-- Logs begin at Sat 2022-05-28 02:37:38 UTC, end at Sat 2022-05-28 03:54:35 UTC. -- May 28 03:26:23.620027 dv-cinder-843361-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 28 03:26:23.629580 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 28 03:26:23.631568 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat May 28 03:26:23 2022] *** May 28 03:26:23.631568 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 28 03:26:23.631568 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 28 03:26:23.631568 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: nodename: dv-cinder-843361-3-iscsi May 28 03:26:23.631568 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: machine: x86_64 May 28 03:26:23.631568 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: clock source: unix May 28 03:26:23.631568 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: pcre jit disabled May 28 03:26:23.632060 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: detected number of CPU cores: 4 May 28 03:26:23.632060 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: current working directory: / May 28 03:26:23.632060 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: detected binary path: /usr/bin/uwsgi-core May 28 03:26:23.632060 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: your processes number limit is 47680 May 28 03:26:23.632060 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: your memory page size is 4096 bytes May 28 03:26:23.632060 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: detected max file descriptor number: 2048 May 28 03:26:23.632060 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: lock engine: pthread robust mutexes May 28 03:26:23.632060 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: thunder lock: enabled May 28 03:26:23.632060 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 28 03:26:23.632060 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 28 03:26:23.677589 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: Python main interpreter initialized at 0x5650fd9d99c0 May 28 03:26:23.677589 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: python threads support enabled May 28 03:26:23.677589 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: your server socket listen backlog is limited to 100 connections May 28 03:26:23.677589 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: your mercy for graceful operations on workers is 90 seconds May 28 03:26:23.677965 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: mapped 403077 bytes (393 KB) for 2 cores May 28 03:26:23.678127 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: *** Operational MODE: preforking *** May 28 03:26:23.678286 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: *** uWSGI is running in multiple interpreter mode *** May 28 03:26:23.678286 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: spawned uWSGI master process (pid: 103705) May 28 03:26:23.678483 dv-cinder-843361-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 28 03:26:23.679639 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: spawned uWSGI worker 1 (pid: 103706, cores: 1) May 28 03:26:23.679639 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: spawned uWSGI worker 2 (pid: 103707, cores: 1) May 28 03:26:23.679639 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 28 03:26:24.847025 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 28 03:26:24.851489 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 28 03:26:25.578869 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:26:25.578869 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: from cryptography.utils import int_from_bytes May 28 03:26:25.579447 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:26:25.579447 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: from cryptography.utils import int_from_bytes May 28 03:26:25.623863 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:26:25.623863 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: from cryptography.utils import int_from_bytes May 28 03:26:25.624192 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:26:25.624192 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: from cryptography.utils import int_from_bytes May 28 03:26:28.194002 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 28 03:26:28.194630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 28 03:26:28.194718 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 28 03:26:28.195030 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 28 03:26:28.195283 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 28 03:26:28.195664 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.195988 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.196304 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.196615 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 28 03:26:28.197010 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.197122 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 28 03:26:28.197217 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.197307 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.197399 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 28 03:26:28.197568 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.197759 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 28 03:26:28.197868 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.198010 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 28 03:26:28.198190 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.198392 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.198478 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.198713 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.198805 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.199028 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.199114 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843361-3-iscsi {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.199340 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.199434 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.199640 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.199732 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.199946 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.200042 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.200258 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.200344 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.200556 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.200640 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.200871 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.200966 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.201143 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.201247 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.201457 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.201679 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.201792 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843361-3-iscsi {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.202080 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.202205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.202381 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.202497 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.202687 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.202811 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.203002 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.203123 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.203307 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.203410 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.203609 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.203702 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.203914 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.203998 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.204213 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.204404 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.204511 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.204709 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.204799 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.205009 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.205221 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843361-3-iscsi {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.205342 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.205561 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.205699 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.205869 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.206008 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.206176 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.206312 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.206578 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.206664 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.206916 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.207053 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.207233 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.207353 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.207537 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.207640 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.207883 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.207983 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843361-3-iscsi {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.208185 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.208285 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.208542 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.208644 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.208808 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.208895 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.209147 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.209289 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.209456 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.209561 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.209752 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.209840 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.210053 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.210152 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.210353 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.210442 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.210664 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.210759 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.210962 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.211055 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.211265 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.211351 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.211509 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.211629 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.211807 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.211944 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.212122 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.212227 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.212532 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.212762 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.212864 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.213080 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.213196 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.213361 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.213468 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.213606 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.213705 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.213921 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.214015 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.214235 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.214324 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.214538 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.214626 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.214841 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.214931 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.215140 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.215258 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.215387 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.215473 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.215633 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.215729 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.215948 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.216047 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.216262 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.216361 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.216564 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.216651 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.216866 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.216953 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.217187 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.217278 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.217489 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.217574 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.217791 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.217877 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.218075 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.218163 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.218374 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.218459 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.41 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.218672 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.218770 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.41 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.218981 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.219066 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.219285 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.219370 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.219598 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.219732 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.219897 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.219989 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.220206 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.220302 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.220516 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.220607 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.41 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.220816 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.220903 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.41 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.221661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.221773 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.221967 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.222053 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.222284 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.222397 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.222586 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.222700 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.222886 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.222982 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.223132 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.223229 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.223428 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.223536 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.223724 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.223823 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.224032 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.224128 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.224329 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.224438 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.224628 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.224730 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.224927 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.225038 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.225188 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.225273 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.225480 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.225568 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.225778 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.225865 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.226088 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.226176 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.226387 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.226481 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.226683 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.226776 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.226980 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.227106 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.227276 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.227366 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.227572 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.227659 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.227870 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.227958 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.228167 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.228252 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.228466 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.228566 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.228762 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.228847 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.229058 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.229174 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.229383 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.229484 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.229678 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.229773 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.229975 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.230060 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.230527 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.230629 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.230826 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.230913 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.231125 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.231220 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.231422 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.231522 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.231716 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.231809 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.232012 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.232105 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.232330 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.232481 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.232612 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.232706 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.232909 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.232996 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.233223 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.233330 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.233523 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.233610 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.233826 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.233918 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.234123 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.234217 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.234420 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.234512 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.234715 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.234802 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.235010 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.235109 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.235301 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.235387 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.235595 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.235686 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.235891 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.235979 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.236186 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.236279 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.236479 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.236566 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.237352 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.237452 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.237656 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.237752 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.237955 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.238042 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.238253 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.238341 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.238552 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.238636 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.238847 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.238935 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.239140 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.239227 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.239448 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.239534 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.239760 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.239855 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.240063 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.240149 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.240365 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.240451 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.240665 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.240759 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.240976 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:26:28.241064 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.241326 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.241422 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.241640 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.241735 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.241967 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.242064 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.242273 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.242358 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.242578 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.242673 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.242886 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.242980 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.243200 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.243294 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.243504 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.243590 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.243819 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.243905 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.244130 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.244215 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.244436 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.244522 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.244745 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.244848 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.245049 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.245207 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.245386 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.245476 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.245698 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.245790 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.246011 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.246096 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.246319 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.246407 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.246629 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.246726 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.246942 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.247030 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.247248 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.247336 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.247557 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.247653 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.247877 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.247965 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.248181 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.248279 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.248484 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.248573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.248789 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.248885 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.249126 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.249226 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.249450 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.249545 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.249756 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.249851 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.250069 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.250155 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.250369 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.250454 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.250674 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.250809 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.250977 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.251070 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.251279 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.251364 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.251583 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.251683 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.251903 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.251989 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.252212 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.252296 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.252541 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.252665 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.252845 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.252930 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.253175 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.253276 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.253479 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.253565 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.253783 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.253869 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.254088 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.254174 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.254385 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.254481 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.254682 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.254770 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.254980 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.255066 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.255284 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.255371 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.255580 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.255678 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.255883 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.255969 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.256186 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.256274 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.256483 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.256577 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.256782 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.256877 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.257081 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.257238 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.257409 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.257496 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.257706 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.257803 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.258010 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.258097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.258311 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.258408 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.258615 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.258712 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.258918 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.259011 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.259218 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.259315 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.259521 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.259621 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.259826 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.259914 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.260123 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.260209 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.260420 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.260507 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.260724 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.260809 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.261023 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.261148 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.261344 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.261438 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.261653 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.261739 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.261956 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.262041 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.262256 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.262343 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.262558 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.262643 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.262856 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.262954 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.263154 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.263241 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.263458 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.263550 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.263767 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.263871 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.264072 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.264161 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.264376 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.264470 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.264681 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.264781 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.264995 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.265089 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.265325 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.265458 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.265623 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.265718 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.265928 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.266016 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.266228 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.266323 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.266528 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.266616 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.266824 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.266911 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.267125 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.267211 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.267420 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.267510 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.267719 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.267806 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.268060 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.268145 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.268357 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.268451 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.268656 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.268742 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.268956 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.269043 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.269288 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.269387 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.269588 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.269673 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.269882 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.269977 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.270180 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.270268 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.270477 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.270571 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.270777 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.270874 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.271093 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.271180 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.271399 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.271486 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.271697 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.271793 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.271997 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.272090 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.272295 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.272381 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.272621 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.272718 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.272921 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.273007 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.273249 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.273340 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.273556 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.273657 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.273861 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.273956 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.274175 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.274262 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.274481 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.274568 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.274794 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.274888 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.275100 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.275197 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.275408 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.275541 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.275709 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.275794 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.276012 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.276098 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.276308 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.276395 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.276618 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.276704 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.276930 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.277015 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.277257 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.277357 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.277577 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.277672 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.277882 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.277979 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.278183 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.278269 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.278481 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.278567 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.278784 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.278883 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.279088 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.279176 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.279392 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.279482 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.279696 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.279792 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.280006 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.280093 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.280308 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.280395 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.280618 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.280713 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.280921 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.281020 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.281249 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.281339 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.281553 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.281652 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.281858 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.281945 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.282155 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.282240 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.282462 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.282548 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.282776 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.282863 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.283082 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.283169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.283389 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.283475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.284217 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.284313 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.284531 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.284617 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.284829 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.284924 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.285149 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.285239 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.285459 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.285559 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.285759 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.285847 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.286057 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.286145 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.286361 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.286456 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.286667 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.286761 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.286968 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.287053 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.287269 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.287363 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.287567 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.287654 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.287866 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.287959 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.288166 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.288261 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.288463 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.288556 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.288758 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.288892 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.289061 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.289206 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.289392 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.289489 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.289693 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.289787 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.290002 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.290097 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.290306 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.290399 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.290600 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.290686 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.290897 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.290984 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.291201 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.291287 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.291503 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.291589 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.291800 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.291895 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.292098 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.292185 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.292401 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.292495 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.292701 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.292795 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.292999 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.293086 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.293338 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.293442 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.293641 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.293727 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.293940 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.294035 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.294244 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.294340 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.294542 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.294632 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.294846 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.294943 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.295151 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.295248 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.295451 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.295537 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.295761 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.295847 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.296066 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.296234 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.296369 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.296465 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.296673 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.296766 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.296977 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.297064 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.297303 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.297391 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.297619 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.297708 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.297919 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.298006 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.298222 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.298308 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.298524 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.298610 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.298824 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.298954 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.299127 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.299254 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.299966 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.300062 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.300285 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.300539 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.300633 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.300872 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.300959 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.301197 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.301339 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.301499 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.301593 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.301803 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.301897 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.302115 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.302203 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.302446 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.302668 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.302754 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.302999 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.303097 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.303317 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.303413 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.303637 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.303725 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.303956 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.304051 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.304260 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.304347 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.304565 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.304652 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.304865 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.304952 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.305189 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.305291 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.305491 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.305588 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.305795 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.305884 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.306094 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.306191 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.306392 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.306480 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.306692 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.306778 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.306989 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.307086 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.307291 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.307377 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.307587 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.307681 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.307889 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.307984 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.308188 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.308275 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.308491 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.308578 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.308791 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.308879 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.309088 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.309245 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.309410 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.309499 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.309708 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.309795 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.310014 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.310100 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.310315 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.310404 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.310632 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.310720 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.310937 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.311025 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.311236 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.311338 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.311535 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.311628 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.311833 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.311920 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.312157 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.312251 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.312457 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.312545 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.312756 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.312842 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.313053 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.313203 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.313385 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.313475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.313691 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.313777 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.313990 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.314086 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.314293 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.314380 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.314597 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.314684 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.314927 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.315021 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.315226 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.315312 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.315522 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.315609 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.315826 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.315961 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.316122 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.316208 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.316427 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.316522 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.316729 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.316817 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.317028 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.317140 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.317351 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.317438 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.317649 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.317737 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.317951 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.318036 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.318248 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.318344 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.318552 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.318640 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.318849 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.318936 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.319147 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.319236 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.319444 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.319530 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.319755 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.319851 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.320036 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.320202 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.320332 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.320419 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.320632 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.320719 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.320926 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.321027 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.321255 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.321354 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.321556 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.321657 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.321851 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.321938 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.322148 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.322233 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.322449 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.322542 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.322747 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.322837 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.323042 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.323129 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.323345 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.323432 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.323649 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.323736 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.323951 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.324040 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.324246 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.324332 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.324550 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.324637 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.324846 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.324941 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.325161 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.325262 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.325461 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.325616 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.325765 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.325856 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.326063 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.326164 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.326360 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.326446 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.326654 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.326739 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.326949 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.327048 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.327244 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.327336 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.327538 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.327624 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.327842 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.327937 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.328136 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.328228 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.328432 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.328526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.328726 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.328813 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.329019 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.329121 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.329351 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.329439 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.329669 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.329765 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.329975 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.330069 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.330273 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.330360 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.330599 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.330689 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.330897 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.330983 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.331200 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.331293 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.331508 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.331596 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.331808 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.331893 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.332121 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.332217 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.332424 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.332521 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.332731 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.332824 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.333030 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.333153 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.333355 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.333486 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.333662 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.333748 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.333965 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.334066 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.334264 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.334357 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.334566 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.334652 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.334874 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.334970 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.335173 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.335260 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.335471 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.335558 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.335771 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.335865 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.336074 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.336215 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.336376 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.336462 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.336675 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.336762 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.336976 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.337064 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.337314 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.337402 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.337614 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.337702 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.337915 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.338000 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.338219 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.338312 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.338521 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.338617 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.338820 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.338907 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.339122 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.339207 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.339424 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.339510 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.339723 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.339809 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.340026 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.340121 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.340331 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.340417 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.340629 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.340718 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.340929 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.341023 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.341250 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.341340 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.341555 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.341643 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.341851 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.341950 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.342169 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.342276 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.342474 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.342582 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.342775 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.342871 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.343078 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.343165 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.343379 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.343466 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.343684 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.343775 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.343987 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.344074 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.344341 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.344426 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.344640 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.344729 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.344988 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.345081 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.345312 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.345403 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.345620 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.345709 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.345927 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.346015 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.346765 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.346919 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.347067 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.347264 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.347352 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.347569 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.347666 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.347981 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.348073 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.348280 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.348366 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.348579 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.348667 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.348884 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.348973 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.349206 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.349310 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.349508 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.349594 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.349813 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.349902 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.350131 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.350219 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.350433 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.350519 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.350731 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.350890 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.351029 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.351197 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.351343 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.351503 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.351643 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.351838 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.351983 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.352135 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.352350 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.352447 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.352659 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.352755 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.352959 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.353046 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.353347 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.353452 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.353668 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.353762 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.353977 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.354084 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.354280 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.354368 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.354589 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.354686 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.354889 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.354976 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.355192 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.355538 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.355755 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.355851 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.356059 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.356146 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.356374 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.356464 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.356692 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.356780 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.356995 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.357090 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.357331 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.357419 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.357632 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.357721 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.357949 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.358037 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.358267 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.358353 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.358576 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.358664 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.358880 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.358975 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.359180 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.359288 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.359480 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.359606 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.359799 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.359913 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.360099 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.360231 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.360402 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.360550 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.360707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.360851 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.361011 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.361169 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.361336 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.361478 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.361725 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.362381 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.362583 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.362697 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.362883 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.362979 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.363203 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.363348 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.363513 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.363610 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.363832 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.363919 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.364150 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.364246 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.367628 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.367755 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.367829 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.367895 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.367960 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.368040 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.368040 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.368213 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.368293 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.368372 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.368450 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.368528 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.368629 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.369548 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.369644 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.369722 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.369856 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.369927 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370004 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370072 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370072 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370190 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370190 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370317 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370384 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370458 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370525 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370589 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370653 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370734 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370800 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370865 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.370929 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371000 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371065 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371138 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371282 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371356 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371423 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371487 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371568 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371637 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371708 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371774 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371839 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371924 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.371992 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.372064 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.372129 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.372194 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.372270 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.372344 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.372411 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.372497 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.374353 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.374425 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.374490 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.374573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.374642 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.374707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.374779 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.374851 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.374917 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.374996 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.375063 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.375128 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.375128 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.375263 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.375352 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.375433 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.375433 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.375584 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.375666 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.375745 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.375913 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.376040 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.376223 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.376326 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.376519 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.376609 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.376821 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.376908 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.377125 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.377216 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.377492 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.377595 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.377794 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.377883 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.380281 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.380392 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.380480 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.380571 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.380654 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.380733 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.380813 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.380893 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.380972 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.381065 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.381166 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.381248 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.381328 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.381408 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.381500 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.381595 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.381676 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.381756 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.381841 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.381922 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382000 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382088 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382168 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382247 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382332 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382420 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382508 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382590 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382669 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382747 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382826 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.382932 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.383135 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.383233 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.383445 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.383753 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.383840 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.384077 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.384173 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.384389 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.384475 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.384707 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.384802 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.385020 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.385139 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.385357 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.385445 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.385668 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.386038 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.386154 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.386276 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.386365 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.386587 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.386682 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.386896 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.391690 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.391778 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.391861 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.391861 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.392006 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.392101 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.392185 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.392264 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.392343 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.392422 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.392501 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.392659 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.392743 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.392832 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.392921 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.393002 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.393095 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.393206 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.393345 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.393439 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.393521 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.393601 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.393690 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.393779 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.393902 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.393998 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.394080 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.394201 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.394343 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.394443 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.394535 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.394618 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.394706 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.394802 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.394883 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.394963 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395030 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395030 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395146 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395236 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395318 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395396 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395474 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395551 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395628 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395715 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395795 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395873 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.395950 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396028 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396131 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396214 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396301 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396381 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396460 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396537 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396628 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396708 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396786 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396786 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.396928 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.397006 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.397092 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.397310 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.397393 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.397472 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.397549 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.397628 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.397725 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.397808 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.397887 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.397971 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.398058 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.398372 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.398460 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.398677 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.398763 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.398979 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.399073 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.399289 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.399378 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.399598 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.399686 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.399901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.399987 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.400208 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.400294 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.400514 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.400607 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.400813 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.400898 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.401128 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.401218 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.401441 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.401538 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.401747 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.401969 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.402058 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.402312 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.402400 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.402627 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.402716 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.402930 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.403017 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.403227 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.403322 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.403533 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.403622 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.403833 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.403921 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.404151 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.404237 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.404449 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.404547 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.404746 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.404833 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.405044 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.405148 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.405378 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.405477 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.405681 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.405768 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.405980 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.406066 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.406280 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.406376 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.406581 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.406668 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.406886 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.406979 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.407187 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.407410 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.407503 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.407583 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.407777 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.407871 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.408074 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.408169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.408373 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.408460 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.409231 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.409323 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.409544 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.409640 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.409847 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.409935 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.410148 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.410243 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.410449 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.410537 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.410748 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.410841 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.411054 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.411149 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.411353 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.411440 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.411657 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.411743 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.411959 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.412045 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.412264 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.412351 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.412564 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.412651 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.412869 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.413122 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.413223 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.413470 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.413558 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.413791 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.413878 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.414120 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.414207 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.414432 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.414518 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.414735 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.414822 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.415039 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.415136 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.415346 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.415541 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.415658 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.415866 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.416001 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.416178 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.416314 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.416487 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.416793 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.416891 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.417121 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.417211 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.417442 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.417554 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.417751 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.417931 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.418062 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.418152 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.418367 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.418454 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.418672 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.418770 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.418976 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.419063 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.419282 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.419368 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.419589 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.419676 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.419893 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.419978 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.420199 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.420291 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.420507 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.420605 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.420813 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.420908 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.421151 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.421243 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.421449 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.421545 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.421756 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.421843 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.422058 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.422152 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.422363 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.422451 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.422666 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.422752 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.422971 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.423065 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.423274 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.423360 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.423584 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.423671 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.423889 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.423983 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.424232 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.424845 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.425079 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.425227 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.425419 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.425508 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.425738 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.425831 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.426045 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.426132 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.426351 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.426438 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.426655 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.426752 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.426959 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.427045 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.427269 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.427356 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.427589 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.427687 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.427892 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.427992 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.428192 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.428278 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.428535 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.428620 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.428859 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.428954 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.429202 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.429296 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.429505 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.429602 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.429805 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.429902 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.430105 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.430192 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.430410 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.430496 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.430724 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.430821 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.431027 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.431122 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.431324 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.431419 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.431621 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.431710 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.431924 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.432011 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.432221 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.432315 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.432519 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.432614 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.432816 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.432903 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.433134 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.433222 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.433442 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.433532 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.433741 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.433828 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.434039 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.434134 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.434334 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.434421 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.434631 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.434725 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.434930 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.435024 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.435228 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.435314 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.435530 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.435615 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.435828 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.435928 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.436125 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.436210 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.436418 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.436504 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.436716 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.436811 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.437013 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.437099 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.437336 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.437423 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.437642 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.437751 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.437945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.438044 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.438243 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.438330 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.438542 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.438631 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.438848 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.438934 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.439148 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.439235 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.439448 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.439537 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.439746 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.439832 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.440087 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.440183 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.440389 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.440483 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.440715 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.440804 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.441023 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.441156 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.441358 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.441446 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.441667 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.441756 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.441974 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.442067 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.442278 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.442364 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.442585 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.442671 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.442905 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.443222 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.443316 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.443534 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.443621 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.443841 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.443927 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.444210 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.444305 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.444473 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.444566 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.444782 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.444877 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.445091 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.445210 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.445421 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.445507 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.445734 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.445829 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.446043 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.446130 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.446351 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.446437 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.446662 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.446830 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.446968 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.447055 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.447276 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.447364 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.447590 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.447679 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.447908 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.448005 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.448220 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.448307 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.448530 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.448624 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.448833 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.448918 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.449175 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.449278 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.449486 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.449573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.449799 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.449899 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.450106 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.450203 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.450413 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.450500 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.450728 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.450815 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.451041 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.451140 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.451350 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.451437 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.451653 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.451739 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.451895 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 28 03:26:28.452046 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.452275 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.452597 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.452940 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.453271 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.453612 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.453937 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.454247 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.454556 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.454856 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:26:28.455096 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 28 03:26:28.510605 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO dbcounter [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Registered counter for database nova_api May 28 03:26:28.513485 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO dbcounter [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] Registered counter for database nova_api May 28 03:26:28.530600 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_db.sqlalchemy.engines [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103707) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:26:28.533200 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG oslo_db.sqlalchemy.engines [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103706) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:26:28.537808 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writer thread running {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:26:28.540189 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG dbcounter [-] [103706] Writer thread running {{(pid=103706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:26:28.593937 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.context [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),023203ac-1a27-4ff9-b011-8825c729816d(cell1) {{(pid=103707) load_cells /opt/stack/nova/nova/context.py:464}} May 28 03:26:28.595462 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.context [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),023203ac-1a27-4ff9-b011-8825c729816d(cell1) {{(pid=103706) load_cells /opt/stack/nova/nova/context.py:464}} May 28 03:26:28.598676 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:26:28.600027 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG oslo_concurrency.lockutils [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103706) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:26:28.600257 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:26:28.601046 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:26:28.601723 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG oslo_concurrency.lockutils [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103706) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:26:28.602500 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG oslo_concurrency.lockutils [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103706) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:26:28.603021 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:26:28.604460 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG oslo_concurrency.lockutils [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103706) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:26:28.604995 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO dbcounter [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Registered counter for database nova_cell0 May 28 03:26:28.606707 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO dbcounter [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] Registered counter for database nova_cell0 May 28 03:26:28.609235 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO dbcounter [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Registered counter for database nova_cell1 May 28 03:26:28.610610 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO dbcounter [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] Registered counter for database nova_cell1 May 28 03:26:28.616717 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_db.sqlalchemy.engines [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103707) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:26:28.618407 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_db.sqlalchemy.engines [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103707) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:26:28.620410 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG oslo_db.sqlalchemy.engines [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103706) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:26:28.622470 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writer thread running {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:26:28.623306 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG oslo_db.sqlalchemy.engines [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103706) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:26:28.626998 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG dbcounter [-] [103706] Writer thread running {{(pid=103706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:26:28.765070 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG dbcounter [-] [103706] Writer thread running {{(pid=103706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:26:28.769867 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writer thread running {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:26:28.775360 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO dbcounter [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] Registered counter for database nova_cell0 May 28 03:26:28.778830 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO dbcounter [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Registered counter for database nova_cell0 May 28 03:26:28.784694 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG oslo_db.sqlalchemy.engines [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103706) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:26:28.786288 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_db.sqlalchemy.engines [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103707) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:26:28.789056 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG dbcounter [-] [103706] Writer thread running {{(pid=103706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:26:28.790567 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writer thread running {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:26:29.226283 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: WARNING keystonemiddleware.auth_token [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 28 03:26:29.233656 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: WARNING keystonemiddleware.auth_token [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 28 03:26:29.255059 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: WARNING keystonemiddleware.auth_token [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 28 03:26:29.262397 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: WARNING keystonemiddleware.auth_token [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 28 03:26:29.570931 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: WARNING keystonemiddleware.auth_token [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 28 03:26:29.576857 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: WARNING keystonemiddleware.auth_token [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 28 03:26:29.582676 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5650fd9d99c0 pid: 103707 (default app) May 28 03:26:29.584714 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:26:29.585901 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] 10.222.0.41 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001739 May 28 03:26:29.586307 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 1/1] 10.222.0.41 () {50 vars in 788 bytes} [Sat May 28 03:26:29 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:26:29.602044 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: WARNING keystonemiddleware.auth_token [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 28 03:26:29.608531 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: WARNING keystonemiddleware.auth_token [None req-bb8ec721-07e6-4930-ba88-fb0673437d0e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 28 03:26:29.613825 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5650fd9d99c0 pid: 103706 (default app) May 28 03:27:50.454640 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:27:50.455735 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001944 May 28 03:27:50.456202 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 2/2] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:27:50 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:27:50.468331 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG dbcounter [-] [103706] Writing DB stats nova_api:SELECT=3 {{(pid=103706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:50.470431 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG dbcounter [-] [103706] Writing DB stats nova_cell1:SELECT=3 {{(pid=103706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:50.471172 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG dbcounter [-] [103706] Writing DB stats nova_cell0:SELECT=3 {{(pid=103706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:50.472343 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG dbcounter [-] [103706] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:50.914577 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-4f5e40f8-0d4c-4ed5-a292-e19138c6ebcb admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:27:50.944259 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-4f5e40f8-0d4c-4ed5-a292-e19138c6ebcb admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.481076 May 28 03:27:50.944917 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 1/3] 10.222.0.41 () {60 vars in 1291 bytes} [Sat May 28 03:27:50 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:27:53.205238 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:27:53.206239 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-cf1f1fed-97d7-4343-98cf-7dfa5d173e9a None None] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001485 May 28 03:27:53.206627 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 3/4] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:27:53 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:27:53.265650 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-147eb102-9226-401b-8b87-3936a11266f5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "ram": 256, "vcpus": 1, "swap": 0, "disk": 1, "id": "c1", "rxtx_factor": 1.0, "name": "cirros256"}} {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:27:53.283760 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:27:53.283760 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: warnings.warn(msg) May 28 03:27:53.378379 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-147eb102-9226-401b-8b87-3936a11266f5 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.165758 May 28 03:27:53.379135 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 2/5] 10.222.0.41 () {64 vars in 1282 bytes} [Sat May 28 03:27:53 2022] POST /compute/v2.1/flavors => generated 407 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:27:53.392631 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writing DB stats nova_api:SELECT=3 {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:53.393872 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writing DB stats nova_cell0:SELECT=3 {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:53.394942 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writing DB stats nova_cell1:SELECT=3 {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:53.395734 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:53.604891 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-75b482c0-8bdd-4ae6-87d0-5078c7f850b2 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:27:53.607643 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:27:53.607643 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: warnings.warn(msg) May 28 03:27:53.736041 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-75b482c0-8bdd-4ae6-87d0-5078c7f850b2 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.347251 May 28 03:27:53.736041 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 4/6] 10.222.0.41 () {64 vars in 1335 bytes} [Sat May 28 03:27:53 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:27:56.012165 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-147eb102-9226-401b-8b87-3936a11266f5 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:27:56.013274 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-147eb102-9226-401b-8b87-3936a11266f5 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001571 May 28 03:27:56.013684 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 3/7] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:27:56 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:27:56.117912 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-c7fd9e9d-dbb0-493a-bc7b-85c662fb3b58 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 5, "vcpus": 1, "id": "d1", "name": "ds512M", "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 512, "rxtx_factor": 1.0, "swap": 0}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:27:56.135628 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:27:56.135628 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: warnings.warn(msg) May 28 03:27:56.229910 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-c7fd9e9d-dbb0-493a-bc7b-85c662fb3b58 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.210233 May 28 03:27:56.230501 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 5/8] 10.222.0.41 () {64 vars in 1282 bytes} [Sat May 28 03:27:56 2022] POST /compute/v2.1/flavors => generated 404 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:27:56.242523 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-49c7dd85-75f8-425f-bac2-314b4aa2b85d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:27:56.244082 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:27:56.244082 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: warnings.warn(msg) May 28 03:27:56.341670 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-49c7dd85-75f8-425f-bac2-314b4aa2b85d admin admin] 10.222.0.41 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.102730 May 28 03:27:56.342541 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 4/9] 10.222.0.41 () {64 vars in 1335 bytes} [Sat May 28 03:27:56 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:27:58.605533 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-c7fd9e9d-dbb0-493a-bc7b-85c662fb3b58 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:27:58.606821 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-c7fd9e9d-dbb0-493a-bc7b-85c662fb3b58 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001518 May 28 03:27:58.612203 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 6/10] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:27:58 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:27:58.674852 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-07a0b10a-f0d7-47a4-b1bd-a74368436aa5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "ram": 1024, "swap": 0, "id": "d2", "name": "ds1G", "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "disk": 10}} {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:27:58.746363 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-07a0b10a-f0d7-47a4-b1bd-a74368436aa5 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.132831 May 28 03:27:58.747496 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 5/11] 10.222.0.41 () {64 vars in 1282 bytes} [Sat May 28 03:27:58 2022] POST /compute/v2.1/flavors => generated 404 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:27:58.759942 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-ca2b8f95-fce2-43ab-b529-c9f389277b14 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:27:58.852955 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-ca2b8f95-fce2-43ab-b529-c9f389277b14 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.097460 May 28 03:27:58.853632 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 7/12] 10.222.0.41 () {64 vars in 1335 bytes} [Sat May 28 03:27:58 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:28:01.131320 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-07a0b10a-f0d7-47a4-b1bd-a74368436aa5 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:01.132346 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-07a0b10a-f0d7-47a4-b1bd-a74368436aa5 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001479 May 28 03:28:01.132761 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 6/13] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:28:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:01.192356 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-39410f73-f63d-49f8-8bec-c94d9b914e91 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds2G", "id": "d3", "rxtx_factor": 1.0, "vcpus": 2, "os-flavor-access:is_public": true, "ram": 2048, "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:01.271173 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-39410f73-f63d-49f8-8bec-c94d9b914e91 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.132387 May 28 03:28:01.271765 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 8/14] 10.222.0.41 () {64 vars in 1282 bytes} [Sat May 28 03:28:01 2022] POST /compute/v2.1/flavors => generated 404 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:28:01.283808 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-c69a8ffd-61c3-4c27-a2e3-2cd5bd52b9b6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:01.358741 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-c69a8ffd-61c3-4c27-a2e3-2cd5bd52b9b6 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.078276 May 28 03:28:01.358893 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 7/15] 10.222.0.41 () {64 vars in 1335 bytes} [Sat May 28 03:28:01 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:28:03.617780 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-39410f73-f63d-49f8-8bec-c94d9b914e91 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:03.618804 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-39410f73-f63d-49f8-8bec-c94d9b914e91 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001464 May 28 03:28:03.619220 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 9/16] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:28:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:03.681056 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-1c0a1fe3-0f58-4aed-9eb4-eca8c77ed7b2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "id": "d4", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "disk": 20, "ram": 4096, "swap": 0, "name": "ds4G", "vcpus": 4}} {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:03.803308 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-1c0a1fe3-0f58-4aed-9eb4-eca8c77ed7b2 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.177844 May 28 03:28:03.804102 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 8/17] 10.222.0.41 () {64 vars in 1282 bytes} [Sat May 28 03:28:03 2022] POST /compute/v2.1/flavors => generated 404 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:28:03.816938 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-549999d0-5d55-4944-a5ea-f48d336098bd admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:03.932135 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-549999d0-5d55-4944-a5ea-f48d336098bd admin admin] 10.222.0.41 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.119535 May 28 03:28:03.932736 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 10/18] 10.222.0.41 () {64 vars in 1335 bytes} [Sat May 28 03:28:03 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:28:06.195511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-1c0a1fe3-0f58-4aed-9eb4-eca8c77ed7b2 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:06.196528 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-1c0a1fe3-0f58-4aed-9eb4-eca8c77ed7b2 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001438 May 28 03:28:06.196928 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 9/19] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:28:06 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:06.258933 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-c2548142-46b4-4263-8818-2c415be37da1 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:06.282117 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-c2548142-46b4-4263-8818-2c415be37da1 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.079154 May 28 03:28:06.282880 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 11/20] 10.222.0.41 () {60 vars in 1291 bytes} [Sat May 28 03:28:06 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:28:08.538116 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-1c0a1fe3-0f58-4aed-9eb4-eca8c77ed7b2 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:08.539140 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-1c0a1fe3-0f58-4aed-9eb4-eca8c77ed7b2 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001468 May 28 03:28:08.539545 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 10/21] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:28:08 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:08.598958 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-4049579e-cf87-4e2d-b1c0-f9033b8ee6df admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "rxtx_factor": 1.0, "ram": 512, "swap": 0, "disk": 1, "os-flavor-access:is_public": true, "name": "m1.tiny", "id": "1", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:08.677071 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-4049579e-cf87-4e2d-b1c0-f9033b8ee6df admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.131212 May 28 03:28:08.678036 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 12/22] 10.222.0.41 () {64 vars in 1282 bytes} [Sat May 28 03:28:08 2022] POST /compute/v2.1/flavors => generated 402 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:28:08.689601 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-4261deca-5ca0-4a3b-85a1-1d59c22074d3 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:08.785254 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-4261deca-5ca0-4a3b-85a1-1d59c22074d3 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.099313 May 28 03:28:08.785986 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 11/23] 10.222.0.41 () {64 vars in 1332 bytes} [Sat May 28 03:28:08 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:28:11.371411 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-4049579e-cf87-4e2d-b1c0-f9033b8ee6df admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:11.372744 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-4049579e-cf87-4e2d-b1c0-f9033b8ee6df admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001955 May 28 03:28:11.373334 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 13/24] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:28:11 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:11.432957 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-0bf23694-2090-4f26-9f07-0568df794199 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "disk": 20, "swap": 0, "rxtx_factor": 1.0, "name": "m1.small", "os-flavor-access:is_public": true, "ram": 2048, "id": "2"}} {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:11.555967 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-0bf23694-2090-4f26-9f07-0568df794199 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.175267 May 28 03:28:11.556454 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 12/25] 10.222.0.41 () {64 vars in 1282 bytes} [Sat May 28 03:28:11 2022] POST /compute/v2.1/flavors => generated 405 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:28:11.568283 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-e3168fad-6aeb-4872-82a0-808ee098d6b5 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:11.667073 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-e3168fad-6aeb-4872-82a0-808ee098d6b5 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.102386 May 28 03:28:11.667664 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 14/26] 10.222.0.41 () {64 vars in 1332 bytes} [Sat May 28 03:28:11 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:28:13.941840 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-0bf23694-2090-4f26-9f07-0568df794199 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:13.942857 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-0bf23694-2090-4f26-9f07-0568df794199 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001501 May 28 03:28:13.943258 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 13/27] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:28:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:14.000512 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-21ab57ef-0b32-4134-864a-9449e63e4d52 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "disk": 40, "swap": 0, "id": "3", "vcpus": 2, "os-flavor-access:is_public": true, "ram": 4096, "rxtx_factor": 1.0, "name": "m1.medium"}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:14.111610 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-21ab57ef-0b32-4134-864a-9449e63e4d52 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.162505 May 28 03:28:14.112315 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 15/28] 10.222.0.41 () {64 vars in 1282 bytes} [Sat May 28 03:28:13 2022] POST /compute/v2.1/flavors => generated 406 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:28:14.124553 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-a2b1418f-ef7d-46b0-bd71-218fbcae56f9 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:14.244982 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-a2b1418f-ef7d-46b0-bd71-218fbcae56f9 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.124091 May 28 03:28:14.245600 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 14/29] 10.222.0.41 () {64 vars in 1332 bytes} [Sat May 28 03:28:14 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:28:16.489965 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-21ab57ef-0b32-4134-864a-9449e63e4d52 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:16.490967 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-21ab57ef-0b32-4134-864a-9449e63e4d52 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 May 28 03:28:16.491371 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 16/30] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:28:16 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:16.551184 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-74d19677-f24a-4d66-9689-878860897fb0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 8192, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.large", "swap": 0, "disk": 80, "id": "4", "vcpus": 4, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:16.651335 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-74d19677-f24a-4d66-9689-878860897fb0 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.154006 May 28 03:28:16.651859 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 15/31] 10.222.0.41 () {64 vars in 1282 bytes} [Sat May 28 03:28:16 2022] POST /compute/v2.1/flavors => generated 405 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:28:16.664165 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-f167432b-ad81-4fb4-8ddf-0840039dbfa7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:16.758194 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-f167432b-ad81-4fb4-8ddf-0840039dbfa7 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.097628 May 28 03:28:16.758972 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 17/32] 10.222.0.41 () {64 vars in 1332 bytes} [Sat May 28 03:28:16 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:28:19.032143 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-74d19677-f24a-4d66-9689-878860897fb0 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:19.033229 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-74d19677-f24a-4d66-9689-878860897fb0 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002051 May 28 03:28:19.033903 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 16/33] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:28:19 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:19.094363 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-fb8efa8f-fe42-4967-8ed5-1b48bee5f59e admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 16384, "name": "m1.xlarge", "vcpus": 8, "rxtx_factor": 1.0, "disk": 160, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "os-flavor-access:is_public": true, "id": "5"}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:19.179711 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-fb8efa8f-fe42-4967-8ed5-1b48bee5f59e admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.139915 May 28 03:28:19.188126 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 18/34] 10.222.0.41 () {64 vars in 1282 bytes} [Sat May 28 03:28:19 2022] POST /compute/v2.1/flavors => generated 408 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:28:19.198550 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-3d5a71a4-fc35-43ab-b6e8-0f77b6427959 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:19.285394 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-3d5a71a4-fc35-43ab-b6e8-0f77b6427959 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.090481 May 28 03:28:19.301711 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 17/35] 10.222.0.41 () {64 vars in 1332 bytes} [Sat May 28 03:28:19 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:28:43.589127 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-fb8efa8f-fe42-4967-8ed5-1b48bee5f59e admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:43.590150 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-fb8efa8f-fe42-4967-8ed5-1b48bee5f59e admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 May 28 03:28:43.590543 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 19/36] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:28:43 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:43.601569 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG dbcounter [-] [103706] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=10,nova_api:SAVEPOINT=5,nova_api:RELEASE=5 {{(pid=103706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:28:43.648704 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-7bc827a3-4cbc-40a9-8d89-9e07a30b2bd9 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:43.665782 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-7bc827a3-4cbc-40a9-8d89-9e07a30b2bd9 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.068770 May 28 03:28:43.666319 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 18/37] 10.222.0.41 () {60 vars in 1291 bytes} [Sat May 28 03:28:43 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:28:44.639736 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-fb8efa8f-fe42-4967-8ed5-1b48bee5f59e admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:44.641335 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-fb8efa8f-fe42-4967-8ed5-1b48bee5f59e admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001968 May 28 03:28:44.641726 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 20/38] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:28:44 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:44.840120 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-73942e68-6f06-4cd4-bf32-8015f9e0fc7d admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:44.846318 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.wsgi [None req-73942e68-6f06-4cd4-bf32-8015f9e0fc7d admin admin] HTTP exception thrown: Flavor 42 could not be found. May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-73942e68-6f06-4cd4-bf32-8015f9e0fc7d admin admin] Request method failure captured: May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Accept: application/json May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Accept-Encoding: gzip, deflate May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Connection: keep-alive May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Content-Length: 0 May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Host: 10.222.0.41 May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Openstack-Api-Version: compute 2.90 May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Openstack-System-Scope: None May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: User-Agent: python-novaclient May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Auth-Token: gAAAAABikZbs5NYy232pxscWxIfZ8LypDWHsvw5r44I0bDxhk2FexZKd5LKmBcKtAZlWbqWeBtka3NY7GwQAaxihl0_3GfyofCjufUw9PR_p-ArSOxQmyw6C3U0hARM_wVEATO70m3ZC5T2-v4WHXc1MucdPNHCYbOgy4b2f6ZphYIsMBJ4cFoc May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Domain-Id: None May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Domain-Name: None May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Identity-Status: Confirmed May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Is-Admin-Project: True May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Openstack-Nova-Api-Version: 2.90 May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Domain-Id: default May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Domain-Name: Default May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Name: admin May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Role: reader,member,admin May 28 03:28:44.849661 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Roles: reader,member,admin May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Tenant: admin May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Tenant-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Tenant-Name: admin May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User: admin May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Domain-Id: default May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Domain-Name: Default May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Id: f454c88a6bba4ab9ba311bbbd1eb0425 May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Name: admin May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: method: > May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: exception: Flavor 42 could not be found. May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: action_args: {'id': '42'} May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:28:44.851405 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:44.852428 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:44.855880 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-73942e68-6f06-4cd4-bf32-8015f9e0fc7d admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103706) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:28:44.857649 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-73942e68-6f06-4cd4-bf32-8015f9e0fc7d admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.062596 May 28 03:28:44.857649 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 19/39] 10.222.0.41 () {60 vars in 1190 bytes} [Sat May 28 03:28:44 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) May 28 03:28:44.861632 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-ffc653ed-11da-4946-a76d-3e0015704a15 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:44.871628 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:28:44.882150 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-ffc653ed-11da-4946-a76d-3e0015704a15 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.024603 May 28 03:28:44.882673 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 21/40] 10.222.0.41 () {60 vars in 1210 bytes} [Sat May 28 03:28:44 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:28:44.896891 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-2727b5fe-bfc6-486a-a75c-1316ee8fed50 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:44.918961 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-2727b5fe-bfc6-486a-a75c-1316ee8fed50 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.025561 May 28 03:28:44.919517 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 20/41] 10.222.0.41 () {60 vars in 1210 bytes} [Sat May 28 03:28:44 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:28:44.929411 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-be76c5fe-2e3b-4b2e-a80b-cb5e32ce9b1b admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:44.935387 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.wsgi [None req-be76c5fe-2e3b-4b2e-a80b-cb5e32ce9b1b admin admin] HTTP exception thrown: Flavor 42 could not be found. May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-be76c5fe-2e3b-4b2e-a80b-cb5e32ce9b1b admin admin] Request method failure captured: May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Accept: application/json May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Accept-Encoding: gzip, deflate May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Connection: keep-alive May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Content-Length: 0 May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Host: 10.222.0.41 May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Openstack-Api-Version: compute 2.90 May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Openstack-System-Scope: None May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: User-Agent: python-novaclient May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Auth-Token: gAAAAABikZbs5NYy232pxscWxIfZ8LypDWHsvw5r44I0bDxhk2FexZKd5LKmBcKtAZlWbqWeBtka3NY7GwQAaxihl0_3GfyofCjufUw9PR_p-ArSOxQmyw6C3U0hARM_wVEATO70m3ZC5T2-v4WHXc1MucdPNHCYbOgy4b2f6ZphYIsMBJ4cFoc May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Domain-Id: None May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Domain-Name: None May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Identity-Status: Confirmed May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Is-Admin-Project: True May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Openstack-Nova-Api-Version: 2.90 May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Domain-Id: default May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Domain-Name: Default May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Name: admin May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Role: reader,member,admin May 28 03:28:44.938707 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Roles: reader,member,admin May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Tenant: admin May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Tenant-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Tenant-Name: admin May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User: admin May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Domain-Id: default May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Domain-Name: Default May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Id: f454c88a6bba4ab9ba311bbbd1eb0425 May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Name: admin May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: method: > May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: exception: Flavor 42 could not be found. May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: action_args: {'id': '42'} May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:28:44.942499 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:44.943286 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:44.944321 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-be76c5fe-2e3b-4b2e-a80b-cb5e32ce9b1b admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103707) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:28:44.945402 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-be76c5fe-2e3b-4b2e-a80b-cb5e32ce9b1b admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.014282 May 28 03:28:44.945402 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 22/42] 10.222.0.41 () {60 vars in 1190 bytes} [Sat May 28 03:28:44 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) May 28 03:28:45.791616 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-2727b5fe-bfc6-486a-a75c-1316ee8fed50 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:45.792631 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-2727b5fe-bfc6-486a-a75c-1316ee8fed50 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001435 May 28 03:28:45.793061 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 21/43] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:28:45 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:45.985715 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-eefdd59c-ffa0-4544-9963-ce7f03691156 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:45.991008 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.wsgi [None req-eefdd59c-ffa0-4544-9963-ce7f03691156 admin admin] HTTP exception thrown: Flavor 84 could not be found. May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-eefdd59c-ffa0-4544-9963-ce7f03691156 admin admin] Request method failure captured: May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Accept: application/json May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Accept-Encoding: gzip, deflate May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Connection: keep-alive May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Content-Length: 0 May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Host: 10.222.0.41 May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Openstack-Api-Version: compute 2.90 May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Openstack-System-Scope: None May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: User-Agent: python-novaclient May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Auth-Token: gAAAAABikZbt-smyggaG7pdifNndTKvkES_Now2rBiu_xofztMoi8QuC2V9bAUy0bnjoRNfwFMkRQO0kwdN4fBy99Vzq11rNeIw179_s1Q8vmrzJ8qKBEw3Xmm8np9cMHTVDp9Guq1Fw6TlqfEnDhgtngSBZZyc_7gSZiewBSzklwKJnVXzFuOY May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Domain-Id: None May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Domain-Name: None May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Identity-Status: Confirmed May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Is-Admin-Project: True May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Openstack-Nova-Api-Version: 2.90 May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Domain-Id: default May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Domain-Name: Default May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Name: admin May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Role: reader,member,admin May 28 03:28:45.992630 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Roles: reader,member,admin May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Tenant: admin May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Tenant-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Tenant-Name: admin May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User: admin May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Domain-Id: default May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Domain-Name: Default May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Id: f454c88a6bba4ab9ba311bbbd1eb0425 May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Name: admin May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: method: > May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: exception: Flavor 84 could not be found. May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: action_args: {'id': '84'} May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:28:45.994184 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:45.995169 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:45.996364 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-eefdd59c-ffa0-4544-9963-ce7f03691156 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103707) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:28:45.997449 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-eefdd59c-ffa0-4544-9963-ce7f03691156 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.059884 May 28 03:28:45.997449 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 23/44] 10.222.0.41 () {60 vars in 1190 bytes} [Sat May 28 03:28:45 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 28 03:28:46.004401 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-ed7d7c22-ee5c-491d-83f5-54783f928e81 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:46.019330 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-ed7d7c22-ee5c-491d-83f5-54783f928e81 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018527 May 28 03:28:46.019855 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 22/45] 10.222.0.41 () {60 vars in 1210 bytes} [Sat May 28 03:28:45 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:28:46.029047 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-b2c76e95-4184-453c-8e77-06241d4dc111 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:46.042933 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-b2c76e95-4184-453c-8e77-06241d4dc111 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017401 May 28 03:28:46.043451 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 24/46] 10.222.0.41 () {60 vars in 1210 bytes} [Sat May 28 03:28:46 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:28:46.053081 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-b553ff55-c7e4-4856-844b-02da6d68a285 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:46.058458 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.wsgi [None req-b553ff55-c7e4-4856-844b-02da6d68a285 admin admin] HTTP exception thrown: Flavor 84 could not be found. May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-b553ff55-c7e4-4856-844b-02da6d68a285 admin admin] Request method failure captured: May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Accept: application/json May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Accept-Encoding: gzip, deflate May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Connection: keep-alive May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Content-Length: 0 May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Host: 10.222.0.41 May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Openstack-Api-Version: compute 2.90 May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Openstack-System-Scope: None May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: User-Agent: python-novaclient May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Auth-Token: gAAAAABikZbt-smyggaG7pdifNndTKvkES_Now2rBiu_xofztMoi8QuC2V9bAUy0bnjoRNfwFMkRQO0kwdN4fBy99Vzq11rNeIw179_s1Q8vmrzJ8qKBEw3Xmm8np9cMHTVDp9Guq1Fw6TlqfEnDhgtngSBZZyc_7gSZiewBSzklwKJnVXzFuOY May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Domain-Id: None May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Domain-Name: None May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Identity-Status: Confirmed May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Is-Admin-Project: True May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Openstack-Nova-Api-Version: 2.90 May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Domain-Id: default May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Domain-Name: Default May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Name: admin May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Role: reader,member,admin May 28 03:28:46.059945 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Roles: reader,member,admin May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Tenant: admin May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Tenant-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Tenant-Name: admin May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User: admin May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Domain-Id: default May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Domain-Name: Default May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Id: f454c88a6bba4ab9ba311bbbd1eb0425 May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Name: admin May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: method: > May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: exception: Flavor 84 could not be found. May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: action_args: {'id': '84'} May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:28:46.061511 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:46.063097 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:46.064475 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-b553ff55-c7e4-4856-844b-02da6d68a285 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103706) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:28:46.065772 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-b553ff55-c7e4-4856-844b-02da6d68a285 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.011771 May 28 03:28:46.065772 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 23/47] 10.222.0.41 () {60 vars in 1190 bytes} [Sat May 28 03:28:46 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 28 03:28:46.931550 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-b2c76e95-4184-453c-8e77-06241d4dc111 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:46.932577 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-b2c76e95-4184-453c-8e77-06241d4dc111 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001421 May 28 03:28:46.932969 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 25/48] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:28:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:47.118824 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-14d33265-6ede-4899-977e-a0f70f06bb38 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:47.124130 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.wsgi [None req-14d33265-6ede-4899-977e-a0f70f06bb38 admin admin] HTTP exception thrown: Flavor 451 could not be found. May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-14d33265-6ede-4899-977e-a0f70f06bb38 admin admin] Request method failure captured: May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Accept: application/json May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Accept-Encoding: gzip, deflate May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Connection: keep-alive May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Content-Length: 0 May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Host: 10.222.0.41 May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Openstack-Api-Version: compute 2.90 May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Openstack-System-Scope: None May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: User-Agent: python-novaclient May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Auth-Token: gAAAAABikZbuMzcky73-D2Thk00Ow1wW9H05VTa-Wypx7nB8_QVNWc24qgi-a2xVifCbKE6cBgW_otH9c9PIQPSFUUiB7Spx-JmQ6FAdpaAf_j2KSmYa-sMx49bq1jbycNrKTCK14gDd7aUSo_pOVceU0Wr1uJlT7aIiK43SeVgug40sil7b4zs May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Domain-Id: None May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Domain-Name: None May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Identity-Status: Confirmed May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Is-Admin-Project: True May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Openstack-Nova-Api-Version: 2.90 May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Domain-Id: default May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Domain-Name: Default May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Project-Name: admin May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Role: reader,member,admin May 28 03:28:47.126179 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Roles: reader,member,admin May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Tenant: admin May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Tenant-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-Tenant-Name: admin May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User: admin May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Domain-Id: default May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Domain-Name: Default May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Id: f454c88a6bba4ab9ba311bbbd1eb0425 May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: X-User-Name: admin May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: method: > May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: exception: Flavor 451 could not be found. May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: action_args: {'id': '451'} May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:28:47.127836 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:47.128901 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: ERROR nova.api.openstack.wsgi May 28 03:28:47.130341 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-14d33265-6ede-4899-977e-a0f70f06bb38 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103706) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:28:47.131572 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-14d33265-6ede-4899-977e-a0f70f06bb38 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.059444 May 28 03:28:47.131572 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 24/49] 10.222.0.41 () {60 vars in 1193 bytes} [Sat May 28 03:28:47 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 28 03:28:47.142162 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-7ca79744-9d81-43d7-ab87-710186b3cbc4 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:47.157522 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-7ca79744-9d81-43d7-ab87-710186b3cbc4 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.022677 May 28 03:28:47.158055 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 26/50] 10.222.0.41 () {60 vars in 1210 bytes} [Sat May 28 03:28:47 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:28:47.168089 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:47.180980 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.016865 May 28 03:28:47.181513 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 25/51] 10.222.0.41 () {60 vars in 1210 bytes} [Sat May 28 03:28:47 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:28:47.190877 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-5e23f109-38fa-4a0f-aab1-7fbe44e8dd52 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:47.196065 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.wsgi [None req-5e23f109-38fa-4a0f-aab1-7fbe44e8dd52 admin admin] HTTP exception thrown: Flavor 451 could not be found. May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-5e23f109-38fa-4a0f-aab1-7fbe44e8dd52 admin admin] Request method failure captured: May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Accept: application/json May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Accept-Encoding: gzip, deflate May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Connection: keep-alive May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Content-Length: 0 May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Host: 10.222.0.41 May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Openstack-Api-Version: compute 2.90 May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Openstack-System-Scope: None May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: User-Agent: python-novaclient May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Auth-Token: gAAAAABikZbuMzcky73-D2Thk00Ow1wW9H05VTa-Wypx7nB8_QVNWc24qgi-a2xVifCbKE6cBgW_otH9c9PIQPSFUUiB7Spx-JmQ6FAdpaAf_j2KSmYa-sMx49bq1jbycNrKTCK14gDd7aUSo_pOVceU0Wr1uJlT7aIiK43SeVgug40sil7b4zs May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Domain-Id: None May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Domain-Name: None May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Identity-Status: Confirmed May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Is-Admin-Project: True May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Openstack-Nova-Api-Version: 2.90 May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Domain-Id: default May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Domain-Name: Default May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Project-Name: admin May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Role: reader,member,admin May 28 03:28:47.197573 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Roles: reader,member,admin May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_cc9d398b29a149a4ab99273f43b67fb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Tenant: admin May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Tenant-Id: cc9d398b29a149a4ab99273f43b67fb2 May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-Tenant-Name: admin May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User: admin May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Domain-Id: default May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Domain-Name: Default May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Id: f454c88a6bba4ab9ba311bbbd1eb0425 May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: X-User-Name: admin May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: method: > May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: exception: Flavor 451 could not be found. May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: action_args: {'id': '451'} May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:28:47.199143 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:47.200205 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: ERROR nova.api.openstack.wsgi May 28 03:28:47.201526 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-5e23f109-38fa-4a0f-aab1-7fbe44e8dd52 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103707) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:28:47.203395 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-5e23f109-38fa-4a0f-aab1-7fbe44e8dd52 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012091 May 28 03:28:47.203395 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 27/52] 10.222.0.41 () {60 vars in 1193 bytes} [Sat May 28 03:28:47 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 28 03:28:48.057737 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:48.058756 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001448 May 28 03:28:48.059153 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 26/53] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:28:48 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:48.246695 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-b20705f9-0958-454f-9dbb-488ff7933dc7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:48.326259 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-b20705f9-0958-454f-9dbb-488ff7933dc7 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.130224 May 28 03:28:48.326802 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 28/54] 10.222.0.41 () {64 vars in 1235 bytes} [Sat May 28 03:28:48 2022] POST /compute/v2.1/flavors => generated 404 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:28:49.186016 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:49.187534 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001466 May 28 03:28:49.187933 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 27/55] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:28:49 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:49.376270 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-eca11c0a-1e0c-4e54-8525-b962b4984fae admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:49.456380 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-eca11c0a-1e0c-4e54-8525-b962b4984fae admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.132782 May 28 03:28:49.456858 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 29/56] 10.222.0.41 () {64 vars in 1235 bytes} [Sat May 28 03:28:49 2022] POST /compute/v2.1/flavors => generated 405 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:28:50.320596 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:50.321663 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001507 May 28 03:28:50.322060 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 28/57] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:28:50 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:50.506971 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-3131b833-5f83-430a-97cb-d3f61abe1196 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:50.604814 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-3131b833-5f83-430a-97cb-d3f61abe1196 admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.147706 May 28 03:28:50.605389 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 30/58] 10.222.0.41 () {64 vars in 1235 bytes} [Sat May 28 03:28:50 2022] POST /compute/v2.1/flavors => generated 407 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:28:51.460822 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:51.461875 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001434 May 28 03:28:51.462275 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 29/59] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:28:51 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:51.648774 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-26a16f96-e67c-4c8a-8baa-d3e165e60e8e admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:28:51.714986 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-26a16f96-e67c-4c8a-8baa-d3e165e60e8e admin admin] 10.222.0.41 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.118287 May 28 03:28:51.715532 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 31/60] 10.222.0.41 () {64 vars in 1235 bytes} [Sat May 28 03:28:51 2022] POST /compute/v2.1/flavors => generated 409 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:28:52.580652 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:52.581699 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001459 May 28 03:28:52.582090 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 30/61] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:28:52 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:28:52.768104 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-881af4f6-ca7e-4ea5-b17f-f681236a8b5c admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:28:52.786943 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-881af4f6-ca7e-4ea5-b17f-f681236a8b5c admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.071022 May 28 03:28:52.787475 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 32/62] 10.222.0.41 () {60 vars in 1202 bytes} [Sat May 28 03:28:52 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:29:54.510427 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:29:54.511439 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-a1a3fb55-1d95-40ae-867c-c4b6b0036a23 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001438 May 28 03:29:54.511835 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 31/63] 10.222.0.41 () {54 vars in 930 bytes} [Sat May 28 03:29:54 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:29:54.519073 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:29:54.570411 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-0b2e0830-61ea-4171-bba8-05ed467fb171 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:29:54.589194 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-0b2e0830-61ea-4171-bba8-05ed467fb171 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.071480 May 28 03:29:54.589758 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 33/64] 10.222.0.41 () {60 vars in 1291 bytes} [Sat May 28 03:29:54 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:29:54.611257 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG dbcounter [-] [103706] Writing DB stats nova_api:SELECT=14 {{(pid=103706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:29:54.615081 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-6cc62fda-81c0-4ffb-bc15-1a75ca3d1c19 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:29:54.622620 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-6cc62fda-81c0-4ffb-bc15-1a75ca3d1c19 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012741 May 28 03:29:54.623145 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 32/65] 10.222.0.41 () {60 vars in 1282 bytes} [Sat May 28 03:29:54 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:29:54.635166 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-cdcf8c64-732b-4594-86c8-4531c9eb573a admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:29:54.643815 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-cdcf8c64-732b-4594-86c8-4531c9eb573a admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012276 May 28 03:29:54.644332 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 34/66] 10.222.0.41 () {60 vars in 1285 bytes} [Sat May 28 03:29:54 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:29:54.655392 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-928ec0a1-153f-4a12-a74e-c251c8451579 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:29:54.662907 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-928ec0a1-153f-4a12-a74e-c251c8451579 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011070 May 28 03:29:54.663423 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 33/67] 10.222.0.41 () {60 vars in 1285 bytes} [Sat May 28 03:29:54 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:29:54.675654 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-53d710ec-8706-4172-8263-fd23c7935401 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:29:54.683224 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-53d710ec-8706-4172-8263-fd23c7935401 admin admin] 10.222.0.41 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011097 May 28 03:29:54.683737 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 35/68] 10.222.0.41 () {60 vars in 1282 bytes} [Sat May 28 03:29:54 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:33:44.263314 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-928ec0a1-153f-4a12-a74e-c251c8451579 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:33:44.264368 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-928ec0a1-153f-4a12-a74e-c251c8451579 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001490 May 28 03:33:44.264800 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 34/69] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:33:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:33:44.420842 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writing DB stats nova_api:SELECT=6 {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:33:44.534220 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-30fe3ddd-7210-472a-9cac-92681afbe755 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:33:44.535896 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:33:44.535896 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: warnings.warn(msg) May 28 03:33:44.537287 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-30fe3ddd-7210-472a-9cac-92681afbe755 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:33:44.537963 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-30fe3ddd-7210-472a-9cac-92681afbe755 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:33:44.538545 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-30fe3ddd-7210-472a-9cac-92681afbe755 admin admin] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:33:44.538975 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-30fe3ddd-7210-472a-9cac-92681afbe755 admin admin] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:33:44.571686 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-30fe3ddd-7210-472a-9cac-92681afbe755 admin admin] 10.222.0.41 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.152219 May 28 03:33:44.572233 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 36/70] 10.222.0.41 () {60 vars in 1193 bytes} [Sat May 28 03:33:44 2022] GET /compute/v2.1/os-services => generated 791 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:33:45.446815 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-928ec0a1-153f-4a12-a74e-c251c8451579 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:33:45.448177 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-928ec0a1-153f-4a12-a74e-c251c8451579 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001778 May 28 03:33:45.448607 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 35/71] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:33:45 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:33:45.642383 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-6f71cb39-b3c3-48af-927f-18d5d9cbf587 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:33:45.644278 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:33:45.644278 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: warnings.warn(msg) May 28 03:33:45.645525 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-6f71cb39-b3c3-48af-927f-18d5d9cbf587 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:33:45.646063 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-6f71cb39-b3c3-48af-927f-18d5d9cbf587 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:33:45.646985 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-6f71cb39-b3c3-48af-927f-18d5d9cbf587 admin admin] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:33:45.647418 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-6f71cb39-b3c3-48af-927f-18d5d9cbf587 admin admin] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:33:45.666849 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-6f71cb39-b3c3-48af-927f-18d5d9cbf587 admin admin] 10.222.0.41 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.077646 May 28 03:33:45.667392 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 37/72] 10.222.0.41 () {60 vars in 1193 bytes} [Sat May 28 03:33:45 2022] GET /compute/v2.1/os-services => generated 791 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:33:56.986859 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-928ec0a1-153f-4a12-a74e-c251c8451579 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:33:56.988500 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-928ec0a1-153f-4a12-a74e-c251c8451579 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002067 May 28 03:33:56.988902 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 36/73] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:33:56 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:33:57.131948 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writing DB stats nova_cell0:SELECT=4 {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:33:57.132667 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writing DB stats nova_cell1:SELECT=4 {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:33:57.133301 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG dbcounter [-] [103707] Writing DB stats nova_api:SELECT=8 {{(pid=103707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:33:57.181347 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-4c00193e-825d-438c-a2e3-482b6986c966 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:33:57.188291 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:33:57.188291 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: warnings.warn(msg) May 28 03:33:57.188291 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-4c00193e-825d-438c-a2e3-482b6986c966 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:33:57.188291 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-4c00193e-825d-438c-a2e3-482b6986c966 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:33:57.188291 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-4c00193e-825d-438c-a2e3-482b6986c966 admin admin] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:33:57.188291 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-4c00193e-825d-438c-a2e3-482b6986c966 admin admin] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:33:57.214138 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-4c00193e-825d-438c-a2e3-482b6986c966 admin admin] 10.222.0.41 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.086461 May 28 03:33:57.214685 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 38/74] 10.222.0.41 () {60 vars in 1193 bytes} [Sat May 28 03:33:57 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:33:58.093776 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: DEBUG nova.api.openstack.wsgi [None req-928ec0a1-153f-4a12-a74e-c251c8451579 admin admin] Calling method '>' {{(pid=103706) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:33:58.094794 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: INFO nova.api.openstack.requestlog [None req-928ec0a1-153f-4a12-a74e-c251c8451579 admin admin] 10.222.0.41 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001463 May 28 03:33:58.095197 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: [pid: 103706|app: 0|req: 37/75] 10.222.0.41 () {56 vars in 1074 bytes} [Sat May 28 03:33:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:33:58.284529 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG nova.api.openstack.wsgi [None req-3ea4de4f-0256-4f28-b4ec-4256ca2219a2 admin admin] Calling method '>' {{(pid=103707) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:33:58.285720 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:33:58.285720 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: warnings.warn(msg) May 28 03:33:58.286926 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-3ea4de4f-0256-4f28-b4ec-4256ca2219a2 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:33:58.287375 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-3ea4de4f-0256-4f28-b4ec-4256ca2219a2 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:33:58.288155 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-3ea4de4f-0256-4f28-b4ec-4256ca2219a2 admin admin] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:33:58.288585 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: DEBUG oslo_concurrency.lockutils [None req-3ea4de4f-0256-4f28-b4ec-4256ca2219a2 admin admin] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103707) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:33:58.310645 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: INFO nova.api.openstack.requestlog [None req-3ea4de4f-0256-4f28-b4ec-4256ca2219a2 admin admin] 10.222.0.41 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.077762 May 28 03:33:58.311189 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: [pid: 103707|app: 0|req: 39/76] 10.222.0.41 () {60 vars in 1193 bytes} [Sat May 28 03:33:58 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:34:55.226201 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: Sat May 28 03:34:55 2022 - graceful shutdown triggered... May 28 03:34:55.226712 dv-cinder-843361-3-iscsi devstack@n-api.service[103707]: Gracefully killing worker 2 (pid: 103707)... May 28 03:34:55.227074 dv-cinder-843361-3-iscsi devstack@n-api.service[103706]: Gracefully killing worker 1 (pid: 103706)... May 28 03:34:55.228118 dv-cinder-843361-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... May 28 03:34:56.226861 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: worker 1 buried after 1 seconds May 28 03:34:56.226861 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: worker 2 buried after 1 seconds May 28 03:34:56.226861 dv-cinder-843361-3-iscsi devstack@n-api.service[103705]: goodbye to uWSGI. May 28 03:34:56.228388 dv-cinder-843361-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. May 28 03:34:56.228946 dv-cinder-843361-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. May 28 03:34:56.231590 dv-cinder-843361-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 28 03:34:56.263758 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 28 03:34:56.269327 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat May 28 03:34:56 2022] *** May 28 03:34:56.269327 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 28 03:34:56.269327 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 28 03:34:56.269327 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: nodename: dv-cinder-843361-3-iscsi May 28 03:34:56.269327 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: machine: x86_64 May 28 03:34:56.269327 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: clock source: unix May 28 03:34:56.269327 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: pcre jit disabled May 28 03:34:56.269327 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: detected number of CPU cores: 4 May 28 03:34:56.269327 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: current working directory: / May 28 03:34:56.269327 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: detected binary path: /usr/bin/uwsgi-core May 28 03:34:56.270134 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: your processes number limit is 47680 May 28 03:34:56.270134 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: your memory page size is 4096 bytes May 28 03:34:56.270134 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: detected max file descriptor number: 2048 May 28 03:34:56.270134 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: lock engine: pthread robust mutexes May 28 03:34:56.270134 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: thunder lock: enabled May 28 03:34:56.270134 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 28 03:34:56.270134 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 28 03:34:56.316807 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: Python main interpreter initialized at 0x56423b36d9c0 May 28 03:34:56.316807 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: python threads support enabled May 28 03:34:56.316807 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: your server socket listen backlog is limited to 100 connections May 28 03:34:56.316807 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: your mercy for graceful operations on workers is 90 seconds May 28 03:34:56.317279 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: mapped 403077 bytes (393 KB) for 2 cores May 28 03:34:56.317279 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: *** Operational MODE: preforking *** May 28 03:34:56.317547 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: *** uWSGI is running in multiple interpreter mode *** May 28 03:34:56.317547 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: spawned uWSGI master process (pid: 124588) May 28 03:34:56.317764 dv-cinder-843361-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 28 03:34:56.320407 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: spawned uWSGI worker 1 (pid: 124604, cores: 1) May 28 03:34:56.320407 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: spawned uWSGI worker 2 (pid: 124605, cores: 1) May 28 03:34:56.320407 dv-cinder-843361-3-iscsi devstack@n-api.service[124588]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 28 03:34:57.499138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 28 03:34:57.502189 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 28 03:34:58.285163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:34:58.285163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: from cryptography.utils import int_from_bytes May 28 03:34:58.285506 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:34:58.285506 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: from cryptography.utils import int_from_bytes May 28 03:34:58.293425 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:34:58.293425 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: from cryptography.utils import int_from_bytes May 28 03:34:58.293764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:34:58.293764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: from cryptography.utils import int_from_bytes May 28 03:35:00.733810 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 28 03:35:00.734509 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 28 03:35:00.734509 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 28 03:35:00.734952 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 28 03:35:00.735331 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 28 03:35:00.736770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.737353 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.737883 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.738403 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.738956 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.739508 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.740081 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.740515 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.740859 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.741161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.741499 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.741831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843361-3-iscsi {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.742161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.742489 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.742792 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.743115 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.743426 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.743762 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.744067 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.744563 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.746470 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.746470 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.746470 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.746470 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.746470 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.746881 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.746961 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 28 03:35:00.747343 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.747433 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 28 03:35:00.747506 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.747506 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.747648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 28 03:35:00.747945 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.748188 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 28 03:35:00.748188 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 28 03:35:00.748621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843361-3-iscsi {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.748621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.749052 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.750520 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.750745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.750974 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.751206 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.751206 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.751664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.751877 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.752095 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.752095 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.752518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.752742 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.752997 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.753243 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.753434 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.753616 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.753808 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.753808 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.754147 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.754313 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.754468 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.754626 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843361-3-iscsi {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.754788 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.754979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.755125 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.755292 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.755451 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.755612 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.755766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.755919 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.756074 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.756261 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.756412 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.756412 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.756702 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.756858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.757003 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.757189 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.758134 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.758327 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.758489 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.758635 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.758829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.759023 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.759181 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.759337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.759337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.759637 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.759842 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.760041 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.760206 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.760360 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.760506 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.760653 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.760805 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.760962 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.761270 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.761426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.761584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.761738 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.761897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.762265 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843361-3-iscsi {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.762432 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.762591 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.762755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.762903 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.763058 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.763215 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.763372 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.763372 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.763664 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.763819 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.763979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.764150 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.764306 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.764463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.764609 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.764760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.764934 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.764934 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.765249 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.41 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.765403 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.765555 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.41 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.765710 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.765868 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.766033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.766189 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.766347 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.766493 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.766647 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.766800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.766946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.767100 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.767295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.767464 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.767913 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.768254 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.768408 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.768581 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.768740 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.768893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.769053 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.769235 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.769384 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.769529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.769673 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.769833 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.769987 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.770133 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.770133 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.770428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.770428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.770743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.770900 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.771047 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.771203 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.771357 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.771513 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.771667 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.771820 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.771979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.772133 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.772295 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.772453 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.772609 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.772761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.772906 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.773051 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.773267 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.773429 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.773584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.773730 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.773880 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.774033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.774189 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.774358 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.774511 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.774668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.774825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.774825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.775108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.775108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.775409 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.775564 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.775718 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.775718 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.776013 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.41 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.776160 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.776314 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.41 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.776467 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.776620 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.776848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.776996 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.777161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.777336 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.777507 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.777655 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.777972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.778153 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.778467 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.778638 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.778802 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.778949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.779101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.779248 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.779401 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.779547 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.779702 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.779849 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.780001 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.780155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.780302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.780459 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.780623 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.780778 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.780925 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.781069 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.781250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.781416 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.781570 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.781723 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.781877 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.782031 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.782183 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.782339 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.782512 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.784566 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.784668 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.784745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.784830 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.784916 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.784991 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.785062 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.785303 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.785402 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.785477 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.785559 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.785639 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.785715 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.785785 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.785860 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.785943 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.786014 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.786083 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.786151 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.786229 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.786306 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.786388 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.786466 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.786537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.786606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.786674 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.786750 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.786833 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.786911 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.786988 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.787057 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.787126 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.787221 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.787302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.787372 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.787448 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.787528 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.787599 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.787667 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.787743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.787840 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.787945 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.788049 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.788049 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.788188 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.788188 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.788331 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.788410 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.788487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.788487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.788640 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.788709 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.788784 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.788862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.788933 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.789001 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.789070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.789070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.789228 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.789307 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.789378 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.789446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.789525 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.789604 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.789682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.789760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.789845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.789916 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.789916 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.789916 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.790110 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.790182 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.790256 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.790331 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.790414 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.790493 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.790574 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.790645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.790720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.790795 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.790870 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.790939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.791015 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.791086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.791163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.791247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.791325 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.791396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.791464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.791550 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.791621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:35:00.791697 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.792433 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.792563 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.792690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.792802 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.792802 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.793026 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.793169 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.793287 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.793472 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.793592 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.793704 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.793813 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.793941 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.794060 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.794181 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.794352 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.794481 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.794618 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.794749 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.794884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.795966 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.796101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.796233 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.796233 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.796480 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.796604 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.796734 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.796864 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.797001 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.797142 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.797259 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.797409 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.797548 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.797691 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.797822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.797946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.798064 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.798270 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.798363 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.798455 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.798570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.798685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.799260 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.799407 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.799407 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.799662 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.799753 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.799959 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.800070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.800332 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.800469 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.800654 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.800757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.800966 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.801173 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.801314 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.814128 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.814305 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.814407 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.814497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.814603 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.814705 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.814793 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.814888 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.814976 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.815122 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.815284 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.815428 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.815428 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.815605 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.815691 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.815786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.815882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.815992 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.816081 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.816175 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.816262 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.816356 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.816356 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.816521 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.816606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.816606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.818664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.818793 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.818793 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.818970 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.819064 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.819064 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.819451 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.819556 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.819651 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.819737 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.819830 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.819916 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.820010 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.820096 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.820234 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.820324 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.820416 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.820525 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.820627 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.820714 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.820798 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.820890 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.820982 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.821068 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.821184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.821273 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.821358 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.821448 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.821541 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.821627 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.821721 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.821807 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.821892 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.821975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.822067 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.822161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.822255 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.822340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.822586 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.822687 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.822861 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.823028 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.823198 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.823353 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.823500 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.823921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.825054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.825753 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.826544 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.826746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.826916 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.827140 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.827320 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.827660 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.827818 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.828002 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.828152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.828306 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.828464 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.828618 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.828812 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.828959 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.829125 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.829306 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.829606 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.829802 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.829967 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.830125 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.830294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.830442 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.830598 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.830753 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.830907 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.831060 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.831237 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.831431 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.831602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.831823 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.831977 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.832121 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.832265 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.832428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.832587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.832808 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.832951 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.833232 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.833380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.833535 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.833694 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.833837 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.834033 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.834033 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.834314 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.834477 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.834629 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.834772 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.834913 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.834913 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.835247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.835406 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.835574 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.835574 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.835866 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.836044 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.836044 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.836044 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.836480 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.836585 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.836672 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.836755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.836852 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.836937 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.837028 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.837207 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.837342 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.837462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.837625 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.837877 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.837974 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838096 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838096 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838363 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838498 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838498 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838653 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838736 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838828 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.838921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.839482 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.839482 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.839482 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.839482 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.840036 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.840135 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.840221 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.840305 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.840395 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.840489 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.840489 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.840714 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.840804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.840886 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.840978 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.841070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.841172 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.841269 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.841269 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.841479 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.841610 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.841709 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.841795 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.841878 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.841960 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.841960 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.842161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.842253 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.842344 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.842435 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.842526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.842617 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.842714 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.842806 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.842931 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.843074 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.843163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.843265 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.843358 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.843450 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.843541 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.843631 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.843726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.843811 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.843906 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.843906 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.844060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.844142 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.844232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.844232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.844437 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.844521 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.844611 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.844611 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.844778 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.844875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.844960 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.845043 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.845177 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.845177 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.850465 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.850636 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.850791 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.850954 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.851049 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.851049 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.851208 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.851302 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.851393 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.851478 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.851620 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.851717 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.851812 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.851896 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.851978 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.852067 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.852150 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.852233 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.852323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.852408 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.852497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.852589 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.852678 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.852770 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.852867 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.852958 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.853042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.853140 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.853226 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.853319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.853413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.853498 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.853580 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.853668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.853752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.853849 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.853934 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.854024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.854107 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.854190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.854272 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.854363 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.854454 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.854544 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.854634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.854717 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.854806 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.854902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.855028 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.855118 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.855118 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.855279 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.855369 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.855461 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.855552 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.855645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.855728 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.855811 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.855911 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.856004 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.856096 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.856186 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.856270 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.856353 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.856444 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.856529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.856653 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.856776 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.856869 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.857044 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.857260 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.857341 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.857485 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.857647 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.857775 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.857891 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.857992 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.858085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.858170 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.858261 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.858352 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.858443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.858528 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.858610 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.858699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.858795 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.858795 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.859090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.859269 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.859367 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.859454 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.859654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.859795 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.859907 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.860016 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.860158 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.860368 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.860816 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.860911 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.860998 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.861185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.861844 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.862015 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.862169 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.862314 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.862456 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.862606 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.862606 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.862896 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.863041 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.863227 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.863375 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.863526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.863669 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.863820 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.863983 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.864127 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.864278 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.864428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.864585 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.864729 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.864877 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.864877 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.865203 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.865362 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.865520 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.865672 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.865840 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.866028 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.866260 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.866260 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.866260 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.866668 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.866835 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.867000 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.867152 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.867335 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.867483 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.867675 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.867810 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.867977 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.868081 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.868162 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.868293 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.868391 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.868482 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.868579 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.868670 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.868761 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.868915 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.869150 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.869254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.869348 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.869434 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.869517 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.869607 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.869759 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.869931 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.870070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.870159 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.870250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.870351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.870437 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.870525 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.870745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.870841 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.870926 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.871008 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.871099 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.871222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.871308 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.871380 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.871449 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.871449 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.871581 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.871735 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.871939 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.872086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.872200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.872400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.872610 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.872733 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.872733 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.872971 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.873083 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.873329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.873559 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.873690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.873836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.873979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.874111 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.874285 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.874447 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.874591 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.874733 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.874883 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.875026 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.875231 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.875408 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.875502 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.875584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.875584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.875584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.875822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.875908 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.875998 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.876083 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.876083 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.876243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.876341 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.876341 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.876517 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.876600 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.876712 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.876800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.876897 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.877086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.877466 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.877632 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.877786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.877945 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.878094 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.878191 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.878277 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.878367 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.878457 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.878551 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.878714 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.878917 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.879009 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.879166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.879277 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.879550 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.879688 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.879835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.879991 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.880176 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.880271 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.880371 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.880506 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.880599 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.880757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.880915 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.880915 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.881361 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.881460 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.881553 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.881700 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.881858 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.882024 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.882162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.882252 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.882390 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.882484 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.882567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.882660 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.882660 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.882874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.882962 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.883053 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.883137 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.883137 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.883299 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.883382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.883472 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.883564 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.883648 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.883828 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.883923 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.884150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.884242 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.884452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.884555 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.884761 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.884860 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.885069 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.899868 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.900025 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.900137 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.900226 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.900309 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.900309 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.900480 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.900572 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.900665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.900750 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.900843 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.900843 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.901062 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.901173 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.901322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.901537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.901677 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.901677 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.911300 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.912091 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.912260 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.912416 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.912560 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.912693 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.912837 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.912978 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.913142 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.913276 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.913387 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.913497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.913618 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.913737 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.913848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.913958 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.914074 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.914193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.914413 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.914532 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.914691 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.914831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.915020 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.915170 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.915421 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.915557 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.915686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.915776 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.915860 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.915954 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.916040 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.916123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.916206 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.916296 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.916389 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.916483 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.916581 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.916666 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.916749 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.916749 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.916900 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.917069 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.917185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.917285 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.917372 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.917455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.917537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.917629 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.917714 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.917714 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.917872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.917963 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.918059 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.918151 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.918235 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.918317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.918406 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.918490 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.918580 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.918671 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.918755 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.918836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.918926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.919017 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.919190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.919284 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.919421 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.919421 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.919668 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.919796 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.919938 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.920074 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.920164 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.920243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.920333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.920431 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.920526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.920610 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.920693 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.920780 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.920871 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.920968 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.921079 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.921237 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.921370 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.921465 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.921550 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.921650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.921736 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.921825 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.921917 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.922007 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.922091 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.922186 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.922271 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.922362 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.922453 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.922537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.922701 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.922831 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.922920 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.922920 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.923082 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.923174 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.923271 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.923356 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.923439 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.923521 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.923660 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.923660 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.924351 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.924458 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.924552 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.924552 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.926036 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.926148 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.926237 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.926330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.926415 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.926499 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.926590 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.926683 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.926768 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.926860 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.926953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927055 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927149 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927234 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927325 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927410 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927501 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927595 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927681 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927854 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.927946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.928257 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.928342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.928425 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.928509 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.928588 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.928681 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.928681 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.928834 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.928912 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.928996 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.929089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.929259 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.929355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.929355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.929504 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.929504 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.929751 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.929845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.929985 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.930115 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.930251 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.930353 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.930439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.930529 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.930625 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.930710 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.930793 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.930884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.930976 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.931060 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.931143 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.931225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.931348 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.931348 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.931568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.931655 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.931738 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.931820 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.931915 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932007 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932098 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932182 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932273 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932364 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932546 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932776 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932859 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932950 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.932950 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.933137 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.933225 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.933308 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.933390 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.933484 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.933579 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.933664 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.933747 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.933837 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.933929 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.934104 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.934193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.934283 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.934368 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.934457 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.934548 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.934644 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.934781 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.934869 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.934961 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.935045 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.935128 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.935223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.935307 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.935397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.935489 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.935568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.935645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.935801 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.935942 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.936091 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.936187 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.936271 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.936363 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.936455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.936547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.936700 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.936831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.936949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.937069 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.937219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.937341 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.937451 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.937568 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.937677 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.937965 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.938223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.938405 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.938507 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.938607 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.938682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.938762 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.938834 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.938904 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.938981 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939144 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939252 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939332 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939488 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939567 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939651 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939729 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939801 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.939940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940074 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940074 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940373 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940451 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940535 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940614 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940685 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940754 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940830 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940914 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.940993 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.941065 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.941255 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.941332 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.941409 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.941493 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.941573 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.941643 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.941719 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.941796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.941866 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.941866 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942011 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942160 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942228 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942309 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942381 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942450 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942586 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942768 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942852 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942924 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.942995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943063 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943145 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943216 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943285 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943361 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943432 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943508 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943508 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943645 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943723 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943794 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.943940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.944570 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.944684 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.944786 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.944919 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.945021 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.945130 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.945230 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.945317 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.945414 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.945499 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.945631 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.945719 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.945810 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.945895 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.945987 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946072 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946163 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946247 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.946946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.947521 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.947521 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.947671 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.947875 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.948155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.948263 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.948544 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.948720 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.948898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.949073 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.949366 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.949466 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.949742 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.949958 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.950059 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.950376 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.950475 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.950687 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.950796 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.951012 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.951126 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.951313 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.951449 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.951725 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.951838 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.951940 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.952096 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.952259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.952423 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.952573 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.952827 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.952924 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.953260 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.953356 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.953522 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.953628 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.953833 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.953997 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.954159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.954286 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.954896 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.955006 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.955090 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.955173 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.955257 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.955329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.955400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.955641 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.955738 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.955833 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.955987 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.956161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.956294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.956532 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.956648 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.956856 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.956988 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.957193 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.957300 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.957519 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.957628 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.957826 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.957929 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.958152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.958255 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.958447 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.958736 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.958834 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.958938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.959067 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.959270 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.959377 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.959639 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.959744 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.959967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.960064 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.960286 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.960394 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.960593 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.960696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.960911 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.961015 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.961248 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.963492 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.963602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.963679 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.963752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.963839 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.963912 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.963993 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964081 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964243 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964315 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964395 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964523 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964622 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964718 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964806 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964906 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964995 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964995 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.964995 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.965241 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.965322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.966043 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.966156 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.966233 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.966305 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.966383 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.966463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.966543 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.966625 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.966698 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.966770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.967612 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.967720 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.967805 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.967879 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.967958 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.968040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.968113 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.968192 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.968273 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.968365 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.969567 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.969720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.969806 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.969888 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.969972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.970053 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.970133 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.970207 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.970279 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.970279 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.970944 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.971028 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.971109 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.971210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.971299 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.972026 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.972121 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.972196 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.972275 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.972348 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.972419 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.972504 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.972578 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.972971 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.973066 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.973516 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.973700 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.973911 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.974012 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.974198 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.974362 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.974527 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.974635 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.974835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.974948 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.975244 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.975355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.976250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.976358 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.976437 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.976510 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.976581 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.976652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.976722 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.976810 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.976810 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.976959 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.977031 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.977137 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.977701 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.977809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.977809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.978109 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.978185 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.978257 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.978339 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.978413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.978727 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.978829 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.978912 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.978986 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.979263 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.979372 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.979593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.979700 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.979777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.979858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.980105 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.980199 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.980386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.980479 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.980697 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.980813 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.980996 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.981187 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.981357 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.981521 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.981689 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.981892 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.982086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.982250 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.982405 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.982557 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.982733 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.982903 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.983059 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.983221 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.983383 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.983544 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.983712 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.983918 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.984093 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.984250 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.984421 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.984577 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.984807 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.984975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.985183 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.987025 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.987215 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.987215 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.987533 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.987725 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.987888 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.988043 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.988043 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.988342 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.988513 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.988668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.988843 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.989006 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.989191 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.989361 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.989525 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.989679 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.989840 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.989995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.990159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.990323 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.990486 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.990650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.990823 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.990987 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.991186 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.991351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.991506 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.991660 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.991821 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.991982 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.992144 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.992308 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.992463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.992625 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.992779 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.992944 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.993165 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.993329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.993651 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.993832 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.994003 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.994163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.994332 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.994487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.994641 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.994795 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.994956 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.995121 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.995285 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.995439 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.995599 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.995599 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.996469 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.996645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.996803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.997138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.997386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.997546 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.997702 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.997876 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.998093 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.998252 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.998414 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.998576 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.998730 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.998899 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.999088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.999277 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.999442 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.999598 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.999598 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:00.999908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.000077 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.000272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.000439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.000601 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.000768 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.000930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.001085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.001330 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.001487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.001487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.001833 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.001990 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.002152 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.002317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.002471 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.002633 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.002796 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.002958 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.003114 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.003267 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.003456 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.003624 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.003809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.003981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.004136 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.004303 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.004465 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.004634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.004789 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.004950 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.005122 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.005296 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 28 03:35:01.005496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.005496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.005496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.005496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.005496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.005496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.005496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.005496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.007351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.007351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.007351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.007351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.007351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.007351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.007351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.007351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.008672 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.008672 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.008672 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.008672 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.008672 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.009468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.009468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.009815 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.009978 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.010196 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.010500 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.010806 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.011111 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.011446 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.011781 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.012099 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.012424 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.012763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.013087 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.013418 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.013741 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.014062 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.014379 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.014690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.015023 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.015359 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.015654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.015968 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.016355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.016861 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.016861 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.018089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.018089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.018419 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.018723 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.019045 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.019372 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.019694 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.020009 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.020342 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.020656 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.020969 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.021310 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.021627 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.021941 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.022261 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.022579 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.022911 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.023233 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.023545 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.023875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.024206 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.025196 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.025196 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.025541 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.025698 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.026170 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.026575 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.027071 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:35:01.027071 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 28 03:35:01.061607 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO dbcounter [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] Registered counter for database nova_api May 28 03:35:01.084500 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_db.sqlalchemy.engines [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124604) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:35:01.088318 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO dbcounter [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] Registered counter for database nova_api May 28 03:35:01.093140 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writer thread running {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:35:01.108420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_db.sqlalchemy.engines [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:35:01.116243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writer thread running {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:35:01.157747 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.context [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),023203ac-1a27-4ff9-b011-8825c729816d(cell1) {{(pid=124604) load_cells /opt/stack/nova/nova/context.py:464}} May 28 03:35:01.162457 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:01.164060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:01.164863 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:01.166958 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:01.168914 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO dbcounter [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] Registered counter for database nova_cell0 May 28 03:35:01.172967 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO dbcounter [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] Registered counter for database nova_cell1 May 28 03:35:01.174676 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.context [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),023203ac-1a27-4ff9-b011-8825c729816d(cell1) {{(pid=124605) load_cells /opt/stack/nova/nova/context.py:464}} May 28 03:35:01.179397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:01.180740 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_db.sqlalchemy.engines [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124604) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:35:01.181137 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:01.182015 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:01.182729 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_db.sqlalchemy.engines [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124604) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:35:01.184245 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:01.186344 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO dbcounter [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] Registered counter for database nova_cell0 May 28 03:35:01.188240 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writer thread running {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:35:01.190705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO dbcounter [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] Registered counter for database nova_cell1 May 28 03:35:01.193551 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writer thread running {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:35:01.198246 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_db.sqlalchemy.engines [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:35:01.200571 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_db.sqlalchemy.engines [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:35:01.208095 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writer thread running {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:35:01.209401 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writer thread running {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:35:01.335838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO dbcounter [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] Registered counter for database nova_cell0 May 28 03:35:01.342509 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_db.sqlalchemy.engines [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124604) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:35:01.347447 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writer thread running {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:35:01.357819 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO dbcounter [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] Registered counter for database nova_cell0 May 28 03:35:01.364952 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_db.sqlalchemy.engines [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:35:01.369189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writer thread running {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:35:01.667854 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: WARNING keystonemiddleware.auth_token [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 28 03:35:01.675058 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: WARNING keystonemiddleware.auth_token [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 28 03:35:01.691127 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: WARNING keystonemiddleware.auth_token [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 28 03:35:01.698826 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: WARNING keystonemiddleware.auth_token [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 28 03:35:02.021231 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: WARNING keystonemiddleware.auth_token [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 28 03:35:02.027239 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: WARNING keystonemiddleware.auth_token [None req-d6664d49-c0b7-4b92-b526-20dd3759610b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 28 03:35:02.032551 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56423b36d9c0 pid: 124604 (default app) May 28 03:35:02.044733 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: WARNING keystonemiddleware.auth_token [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 28 03:35:02.051386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: WARNING keystonemiddleware.auth_token [None req-51e00a44-4f12-44bb-9550-0d06bf40b74e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 28 03:35:02.056702 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56423b36d9c0 pid: 124605 (default app) May 28 03:35:44.647234 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=3 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:44.649227 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=3 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:44.650252 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=3 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:44.651349 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=3 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:45.929872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-54e742ba-2010-4fb9-a3ff-0a30c5dcabc3 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] Action: 'create', calling method: .version_select at 0x7feb2dda5c10>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-28666878", "metadata": {"Type": "work"}}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:35:47.869580 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-54e742ba-2010-4fb9-a3ff-0a30c5dcabc3 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] 10.222.0.41 "POST /compute/v2.1/os-volumes" status: 200 len: 351 microversion: 2.1 time: 3.147931 May 28 03:35:47.869580 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 1/1] 10.222.0.41 () {60 vars in 1155 bytes} [Sat May 28 03:35:44 2022] POST /compute/v2.1/os-volumes => generated 351 bytes in 3151 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) May 28 03:35:48.129528 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6b091353-4ad9-4e41-bda1-08cf308e1de9 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] Calling method '.version_select at 0x7feb2dd6a5e0>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:48.177995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=3 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:48.179033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=3 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:48.181364 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=3 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:48.181746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=3 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:48.501974 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6b091353-4ad9-4e41-bda1-08cf308e1de9 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee" status: 200 len: 351 microversion: 2.1 time: 0.714623 May 28 03:35:48.502692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 1/2] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:35:47 2022] GET /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee => generated 351 bytes in 719 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:35:48.869756 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-41bbbacb-c311-498a-9f6a-6d02d4c674a3 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] Calling method '.version_select at 0x7feb2dbf5b80>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:48.947156 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-41bbbacb-c311-498a-9f6a-6d02d4c674a3 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/0fbd6d6a-7da0-43cd-8e07-3f153c1ae6e6" status: 200 len: 351 microversion: 2.1 time: 0.082176 May 28 03:35:48.960846 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 2/3] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:35:48 2022] GET /compute/v2.1/os-volumes/0fbd6d6a-7da0-43cd-8e07-3f153c1ae6e6 => generated 351 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:35:49.456024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-918853302", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "networks": [{"uuid": "4150b5cc-830a-4fc6-9434-807c7104e296"}]}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:35:49.516540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1b48e2c7-8528-47b0-aede-5df8b35ac52c tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] Calling method '.version_select at 0x7feb2dbf5d30>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:49.670701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.network.neutron [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] validate_networks() for [('4150b5cc-830a-4fc6-9434-807c7104e296', None, None, None, None, None)] {{(pid=124605) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:35:50.000455 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1b48e2c7-8528-47b0-aede-5df8b35ac52c tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee" status: 200 len: 352 microversion: 2.1 time: 0.489199 May 28 03:35:50.001534 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 3/4] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:35:49 2022] GET /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee => generated 352 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:35:50.013202 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e0a2860e-a047-4c44-8498-84cb4fde1fe1 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] Calling method '.version_select at 0x7feb2dc07430>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:50.218493 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e0a2860e-a047-4c44-8498-84cb4fde1fe1 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee" status: 200 len: 352 microversion: 2.1 time: 0.209146 May 28 03:35:50.218833 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 4/5] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:35:50 2022] GET /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee => generated 352 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:35:50.237008 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-08bd1130-4f5a-46f1-8adf-328bb9d9de03 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] Calling method '.version_select at 0x7feb2dc07af0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:50.544441 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-08bd1130-4f5a-46f1-8adf-328bb9d9de03 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] 10.222.0.41 "DELETE /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee" status: 202 len: 0 microversion: 2.1 time: 0.313437 May 28 03:35:50.547595 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 5/6] 10.222.0.41 () {58 vars in 1247 bytes} [Sat May 28 03:35:50 2022] DELETE /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee => generated 0 bytes in 317 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:35:50.567167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ba3f8087-e63c-4531-8eaa-c79606e2ed96 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] Calling method '.version_select at 0x7feb2dc07ca0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:50.686589 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ba3f8087-e63c-4531-8eaa-c79606e2ed96 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee" status: 200 len: 351 microversion: 2.1 time: 0.134431 May 28 03:35:50.687418 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 6/7] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:35:50 2022] GET /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee => generated 351 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:35:51.028476 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Flavor limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:35:51.028476 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Image limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:35:51.028997 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Flavor pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:35:51.029411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Image pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:35:51.029952 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:35:51.062996 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:51.065571 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:51.069625 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:51.070223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:51.160248 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Getting quotas for project fc3ad7416d27411cb8ddbce0a3da18eb. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:35:51.184367 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Getting quotas for user 46e48b861bc2498c9670de030c783f54 and project fc3ad7416d27411cb8ddbce0a3da18eb. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:35:51.201015 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2c2c1fb3-613c-4017-a1d8-b7cc92fd4277 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] Calling method '.version_select at 0x7feb2dc074c0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:51.220708 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Going to run 1 instances... {{(pid=124605) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:35:51.236865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:35:51.317744 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2c2c1fb3-613c-4017-a1d8-b7cc92fd4277 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/50a11600-a85f-40f4-8b57-40b84f26f401" status: 200 len: 351 microversion: 2.1 time: 0.120488 May 28 03:35:51.318912 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 7/8] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:35:51 2022] GET /compute/v2.1/os-volumes/50a11600-a85f-40f4-8b57-40b84f26f401 => generated 351 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:35:51.511802 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-035304b1-e428-4bf2-9332-6558513d69c0 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.161677 May 28 03:35:51.519740 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 2/9] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:35:49 2022] POST /compute/v2.1/servers => generated 380 bytes in 2178 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:35:51.526247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-43e20cbc-b76e-4f57-ae19-c267a9245efa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:51.529413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-43e20cbc-b76e-4f57-ae19-c267a9245efa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:35:51.707089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9bfa5378-d4ad-4059-bda0-3e424c2d7529 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] Calling method '.version_select at 0x7feb2d305790>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:51.814575 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-43e20cbc-b76e-4f57-ae19-c267a9245efa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:35:51.844086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-43e20cbc-b76e-4f57-ae19-c267a9245efa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1214 microversion: 2.1 time: 0.323040 May 28 03:35:51.846166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 8/10] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:35:51 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1214 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:35:52.169801 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-9bfa5378-d4ad-4059-bda0-3e424c2d7529 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] HTTP exception thrown: Volume 6af10457-821f-40ec-9219-5b26846fffee could not be found. May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9bfa5378-d4ad-4059-bda0-3e424c2d7529 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] Request method failure captured: May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee HTTP/1.1 May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZiQwFa7IcNiEYiub-162gL66_gO2ynP6Nv7MqhvXg_-USkYz3xFAyygXfAXkAc3rN8uCWKbyojWB4aLyVa7ifZ_kVdumUULjdlZSClYKw1RvDugv3g5fpGm5ZrLNejZDQVzt4fz7IYlgW5qVnuUC2I9ft5EdVujrn9v_VEyt9zns18 May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: c547fb861a0e4888b6bef6c30c5f0c73 May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-VolumesGetTestJSON-1279529229 May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:35:52.174879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/c547fb861a0e4888b6bef6c30c5f0c73"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c547fb861a0e4888b6bef6c30c5f0c73"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c547fb861a0e4888b6bef6c30c5f0c73"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_c547fb861a0e4888b6bef6c30c5f0c73"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-VolumesGetTestJSON-1279529229 May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: c547fb861a0e4888b6bef6c30c5f0c73 May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-VolumesGetTestJSON-1279529229 May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-VolumesGetTestJSON-1279529229-project May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 4c9b89c578a34597b143cb9294946300 May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-VolumesGetTestJSON-1279529229-project May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Volume 6af10457-821f-40ec-9219-5b26846fffee could not be found. May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '6af10457-821f-40ec-9219-5b26846fffee'} May 28 03:35:52.176189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi item = cinderclient( May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 6af10457-821f-40ec-9219-5b26846fffee could not be found. (HTTP 404) (Request-ID: req-f2da5a42-48fd-4ef7-8c2e-f52b2b43d58b) May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:35:52.176872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi item = cinderclient( May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 28 03:35:52.177777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 6af10457-821f-40ec-9219-5b26846fffee could not be found. May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 28 03:35:52.178762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 6af10457-821f-40ec-9219-5b26846fffee could not be found. May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 6af10457-821f-40ec-9219-5b26846fffee could not be found. May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9bfa5378-d4ad-4059-bda0-3e424c2d7529 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] Returning 404 to user: Volume 6af10457-821f-40ec-9219-5b26846fffee could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:35:52.179621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9bfa5378-d4ad-4059-bda0-3e424c2d7529 tempest-VolumesGetTestJSON-1279529229 tempest-VolumesGetTestJSON-1279529229-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee" status: 404 len: 109 microversion: 2.1 time: 0.479234 May 28 03:35:52.192996 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 3/11] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:35:51 2022] GET /compute/v2.1/os-volumes/6af10457-821f-40ec-9219-5b26846fffee => generated 109 bytes in 485 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:35:52.198666 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-29aa9d71-3f6f-40c1-8aa2-94540b7c22b7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Action: 'create', calling method: .version_select at 0x7feb2dc11940>, body: {"keypair": {"name": "tempest-keypair-2087163104", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBALLF1QgtPhXS28/zHeAR+xlguvrDDYJJbouwv/QHxZyKN0eZJ3h4ENYVcayYEw3X4Y5uAHdLry4VgNjfHMml51VrWY8EB+j0RqtKI7SNY+TPfDD0WToWDjg8DZsU4552Q=="}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:35:52.310143 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-29aa9d71-3f6f-40c1-8aa2-94540b7c22b7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Getting quotas for project 3611c72fb0b8472da98ba9b0c0d86843. Resources: {'key_pairs'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:35:52.326946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-29aa9d71-3f6f-40c1-8aa2-94540b7c22b7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Getting quotas for user e0eb0ab5585747d098bbb0b18770f4c5 and project 3611c72fb0b8472da98ba9b0c0d86843. Resources: {'key_pairs'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:35:52.532943 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-29aa9d71-3f6f-40c1-8aa2-94540b7c22b7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.510111 May 28 03:35:52.539527 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 9/12] 10.222.0.41 () {62 vars in 1199 bytes} [Sat May 28 03:35:52 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 517 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) May 28 03:35:52.865142 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-31007c29-c35e-4500-9f89-e1e99a698ee3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:52.865834 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-31007c29-c35e-4500-9f89-e1e99a698ee3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:35:53.132063 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-31007c29-c35e-4500-9f89-e1e99a698ee3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:35:53.160229 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-31007c29-c35e-4500-9f89-e1e99a698ee3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1214 microversion: 2.1 time: 0.299911 May 28 03:35:53.161475 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 4/13] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:35:52 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1214 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:35:53.427626 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-850bb676-e2da-4de2-b219-3d7733a7e436 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] Calling method '.version_select at 0x7feb2db2fca0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:53.545075 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-850bb676-e2da-4de2-b219-3d7733a7e436 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/c850a2a0-812f-4fb6-ae6d-88978adedb87" status: 200 len: 350 microversion: 2.1 time: 0.123192 May 28 03:35:53.546101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 10/14] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:35:53 2022] GET /compute/v2.1/os-volumes/c850a2a0-812f-4fb6-ae6d-88978adedb87 => generated 350 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:35:53.558261 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a314cb88-1cef-4d28-96e4-b669422b073b tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] Calling method '.version_select at 0x7feb2ca7e790>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:53.688282 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a314cb88-1cef-4d28-96e4-b669422b073b tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] 10.222.0.41 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.134135 May 28 03:35:53.689352 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 5/15] 10.222.0.41 () {58 vars in 1133 bytes} [Sat May 28 03:35:53 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:35:53.708451 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bbf5cafd-5204-464d-b725-9c7e04ec4027 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] Calling method '.version_select at 0x7feb2db2fe50>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:53.865858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bbf5cafd-5204-464d-b725-9c7e04ec4027 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] 10.222.0.41 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.161957 May 28 03:35:53.866486 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 11/16] 10.222.0.41 () {58 vars in 1148 bytes} [Sat May 28 03:35:53 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:35:53.879566 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-25e8205f-2f0c-4deb-b34d-4e884bf49a88 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] Calling method '.version_select at 0x7feb2ca2c940>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:54.086479 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-25e8205f-2f0c-4deb-b34d-4e884bf49a88 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] 10.222.0.41 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.211557 May 28 03:35:54.093283 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 6/17] 10.222.0.41 () {58 vars in 1133 bytes} [Sat May 28 03:35:53 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:35:54.110152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3ccc90f3-693e-40c7-b39a-9afdd1a2a1ef tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] Calling method '.version_select at 0x7feb2db63670>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:54.180375 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-51684ffe-063f-422a-a44b-e08d08f54d2a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:54.181120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-51684ffe-063f-422a-a44b-e08d08f54d2a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:35:54.275632 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3ccc90f3-693e-40c7-b39a-9afdd1a2a1ef tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] 10.222.0.41 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.180169 May 28 03:35:54.282909 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 12/18] 10.222.0.41 () {58 vars in 1166 bytes} [Sat May 28 03:35:54 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:35:54.291811 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e91a509f-7538-4c03-b84a-6d90d6d378cb tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] Calling method '.version_select at 0x7feb2d2ec310>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:54.410240 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-51684ffe-063f-422a-a44b-e08d08f54d2a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:35:54.432638 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-51684ffe-063f-422a-a44b-e08d08f54d2a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1214 microversion: 2.1 time: 0.256033 May 28 03:35:54.446494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 7/19] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:35:54 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1214 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:35:54.515067 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e91a509f-7538-4c03-b84a-6d90d6d378cb tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.230481 May 28 03:35:54.529267 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 13/20] 10.222.0.41 () {58 vars in 1169 bytes} [Sat May 28 03:35:54 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:35:54.531547 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f809d508-c194-48ec-834e-500644a2bec5 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] Calling method '.version_select at 0x7feb2ca2cf70>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:54.654736 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f809d508-c194-48ec-834e-500644a2bec5 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.129016 May 28 03:35:54.665219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 8/21] 10.222.0.41 () {58 vars in 1154 bytes} [Sat May 28 03:35:54 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:35:54.666887 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4d38a81e-f6b7-4a68-94eb-de4885135200 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] Calling method '.version_select at 0x7feb2db2fdc0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:54.780150 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4d38a81e-f6b7-4a68-94eb-de4885135200 tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.116943 May 28 03:35:54.780979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 14/22] 10.222.0.41 () {58 vars in 1187 bytes} [Sat May 28 03:35:54 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:35:54.793475 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3894bd9b-241d-474e-8982-eb89fa71fcac tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] Calling method '.version_select at 0x7feb2cae2f70>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:54.904211 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3894bd9b-241d-474e-8982-eb89fa71fcac tempest-VolumesTestJSON-587582939 tempest-VolumesTestJSON-587582939-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.114758 May 28 03:35:54.905503 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 9/23] 10.222.0.41 () {58 vars in 1154 bytes} [Sat May 28 03:35:54 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:35:55.451950 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-473a6e30-d95b-451a-b155-629760776a0d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:55.453002 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-473a6e30-d95b-451a-b155-629760776a0d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:35:55.470059 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-473a6e30-d95b-451a-b155-629760776a0d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:55.476908 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-473a6e30-d95b-451a-b155-629760776a0d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:55.478093 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-473a6e30-d95b-451a-b155-629760776a0d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:55.478733 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-473a6e30-d95b-451a-b155-629760776a0d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:55.882911 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-473a6e30-d95b-451a-b155-629760776a0d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:35:55.920697 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-473a6e30-d95b-451a-b155-629760776a0d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1218 microversion: 2.1 time: 0.473325 May 28 03:35:55.930954 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 15/24] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:35:55 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1218 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:35:56.939653 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0363eb15-e3b5-439b-bc3a-5c685a2c12e3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:56.941910 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0363eb15-e3b5-439b-bc3a-5c685a2c12e3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:35:56.986220 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0363eb15-e3b5-439b-bc3a-5c685a2c12e3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:56.993411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0363eb15-e3b5-439b-bc3a-5c685a2c12e3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:56.994253 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0363eb15-e3b5-439b-bc3a-5c685a2c12e3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:56.994702 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0363eb15-e3b5-439b-bc3a-5c685a2c12e3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:57.917276 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0363eb15-e3b5-439b-bc3a-5c685a2c12e3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:35:57.952217 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0363eb15-e3b5-439b-bc3a-5c685a2c12e3 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1266 microversion: 2.1 time: 1.016131 May 28 03:35:57.952865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 10/25] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:35:56 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1266 bytes in 1017 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:35:58.972271 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-492f3a90-39c4-4913-a523-5b6a75525340 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:35:58.972966 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-492f3a90-39c4-4913-a523-5b6a75525340 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:35:58.980048 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-492f3a90-39c4-4913-a523-5b6a75525340 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:58.980505 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-492f3a90-39c4-4913-a523-5b6a75525340 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:59.005005 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-492f3a90-39c4-4913-a523-5b6a75525340 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:35:59.008607 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-492f3a90-39c4-4913-a523-5b6a75525340 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.024s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:35:59.116231 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1195433255", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f47d9414-85e4-4030-814e-0a5133c3d4b0"}], "security_groups": [{"name": "tempest-securitygroup--1419239839"}], "key_name": "tempest-keypair-2087163104", "user_data": "}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:35:59.310245 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.network.neutron [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] validate_networks() for [('f47d9414-85e4-4030-814e-0a5133c3d4b0', None, None, None, None, None)] {{(pid=124605) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:35:59.318999 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-492f3a90-39c4-4913-a523-5b6a75525340 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:35:59.335857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-492f3a90-39c4-4913-a523-5b6a75525340 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1272 microversion: 2.1 time: 0.365772 May 28 03:35:59.336320 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 16/26] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:35:58 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1272 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:00.352138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f0677046-e453-4cea-909e-3fd2c6f6e684 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:00.352722 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f0677046-e453-4cea-909e-3fd2c6f6e684 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:00.372637 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f0677046-e453-4cea-909e-3fd2c6f6e684 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:00.379154 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f0677046-e453-4cea-909e-3fd2c6f6e684 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:00.380021 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f0677046-e453-4cea-909e-3fd2c6f6e684 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:00.380486 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f0677046-e453-4cea-909e-3fd2c6f6e684 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:01.020325 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Flavor limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:36:01.021812 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Image limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:36:01.022457 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Flavor pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:36:01.022893 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Image pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:36:01.023573 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:36:01.045023 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:01.048107 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:01.049185 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:01.052039 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:01.165577 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Getting quotas for project 3611c72fb0b8472da98ba9b0c0d86843. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:36:01.213552 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Getting quotas for user e0eb0ab5585747d098bbb0b18770f4c5 and project 3611c72fb0b8472da98ba9b0c0d86843. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:36:01.235711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Going to run 1 instances... {{(pid=124605) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:36:01.242465 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:36:01.400128 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f0677046-e453-4cea-909e-3fd2c6f6e684 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:01.400128 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f0677046-e453-4cea-909e-3fd2c6f6e684 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1272 microversion: 2.1 time: 1.039240 May 28 03:36:01.400128 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 17/27] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:00 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1272 bytes in 1041 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:01.754570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c5c76c60-34d2-422b-a38b-07c248b0252e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 2.635559 May 28 03:36:01.757134 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 11/28] 10.222.0.41 () {62 vars in 1187 bytes} [Sat May 28 03:35:59 2022] POST /compute/v2.1/servers => generated 402 bytes in 2647 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) May 28 03:36:01.775984 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6cef2956-b26d-4378-a1a9-e9bff21c108e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:01.777139 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-6cef2956-b26d-4378-a1a9-e9bff21c108e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:02.170353 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6cef2956-b26d-4378-a1a9-e9bff21c108e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:02.181929 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6cef2956-b26d-4378-a1a9-e9bff21c108e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:02.185298 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6cef2956-b26d-4378-a1a9-e9bff21c108e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:02.185887 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6cef2956-b26d-4378-a1a9-e9bff21c108e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1282 microversion: 2.20 time: 0.420249 May 28 03:36:02.197976 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 18/29] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:01 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1282 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:02.414445 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e8fcd08a-bf56-48c9-bf61-50497f0e9427 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:02.425776 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e8fcd08a-bf56-48c9-bf61-50497f0e9427 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:02.456963 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e8fcd08a-bf56-48c9-bf61-50497f0e9427 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:02.464711 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e8fcd08a-bf56-48c9-bf61-50497f0e9427 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:02.470755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e8fcd08a-bf56-48c9-bf61-50497f0e9427 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:02.478529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e8fcd08a-bf56-48c9-bf61-50497f0e9427 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:02.579657 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ee93a84d-2f69-498f-b2d9-c0be6e39c7f1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: .version_select at 0x7feb2c9d4940>, body: {"keypair": {"name": "tempest-keypair-2087622025", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFlrLvBT5cBepx63kcUjDsmEC9+Cu0dcglT2SNrvnKAgjtxY3+VeNyq9T3hmPcZKXvuroHynKlviLTCaqrL4ep81U/RcjJLwknVFKOE5UL2sYXbCtSaMAJEjeT5Z7Ui+oA=="}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:02.674314 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-ee93a84d-2f69-498f-b2d9-c0be6e39c7f1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:36:02.697201 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-ee93a84d-2f69-498f-b2d9-c0be6e39c7f1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for user 9d420e4b554e4deab21b89094e0547ca and project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:36:02.877735 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ee93a84d-2f69-498f-b2d9-c0be6e39c7f1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.708792 May 28 03:36:02.879709 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 12/30] 10.222.0.41 () {60 vars in 1158 bytes} [Sat May 28 03:36:02 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 711 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:36:02.920550 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e8fcd08a-bf56-48c9-bf61-50497f0e9427 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:02.934593 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e8fcd08a-bf56-48c9-bf61-50497f0e9427 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1272 microversion: 2.1 time: 0.532837 May 28 03:36:02.938443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 19/31] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:02 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1272 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:03.211038 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-808abbf7-280c-4397-8a92-eb5e8322495d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:03.211674 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-808abbf7-280c-4397-8a92-eb5e8322495d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:03.397141 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-808abbf7-280c-4397-8a92-eb5e8322495d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:03.469576 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-808abbf7-280c-4397-8a92-eb5e8322495d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:03.488533 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-808abbf7-280c-4397-8a92-eb5e8322495d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:03.503766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-808abbf7-280c-4397-8a92-eb5e8322495d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1282 microversion: 2.20 time: 0.299552 May 28 03:36:03.515378 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 20/32] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:03 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1282 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:03.663988 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8e491205-2abd-4ca0-885e-0e40f9c3f91d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Action: 'create', calling method: .version_select at 0x7feb2c9d4d30>, body: {"keypair": {"name": "tempest-keypair-447830529", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFkM+PA4sTasMMCfKs/Vb+nzdt7uOFGRwnyOpAaFLiRt6LAAwGcpr3A1un+EzC18EVJ3sevnmxViS8IbjUb3PWYSRqoeZjKbGAJJqsCXMrP1xW5eFJ76x/S89NO6mlnpMQ=="}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:03.737085 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-8e491205-2abd-4ca0-885e-0e40f9c3f91d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Getting quotas for project 70eaf602c8db4ec1bfb26e5eb3b13c82. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:36:03.753733 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-8e491205-2abd-4ca0-885e-0e40f9c3f91d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Getting quotas for user dc6a3335562448d1934fa18e8f51b119 and project 70eaf602c8db4ec1bfb26e5eb3b13c82. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:36:03.927208 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8e491205-2abd-4ca0-885e-0e40f9c3f91d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.726930 May 28 03:36:03.927373 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 13/33] 10.222.0.41 () {60 vars in 1158 bytes} [Sat May 28 03:36:03 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:03.969880 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-db8b6c4a-27b3-4c83-9f34-81e6d8dd2736 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:03.970510 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-db8b6c4a-27b3-4c83-9f34-81e6d8dd2736 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:03.993849 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-db8b6c4a-27b3-4c83-9f34-81e6d8dd2736 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:03.994782 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-db8b6c4a-27b3-4c83-9f34-81e6d8dd2736 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:04.000010 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-db8b6c4a-27b3-4c83-9f34-81e6d8dd2736 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:04.000604 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-db8b6c4a-27b3-4c83-9f34-81e6d8dd2736 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:04.525160 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7df6f491-90c2-49fa-a9ab-b1cb5167addb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:04.525160 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7df6f491-90c2-49fa-a9ab-b1cb5167addb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:04.660976 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-db8b6c4a-27b3-4c83-9f34-81e6d8dd2736 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:04.677754 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-db8b6c4a-27b3-4c83-9f34-81e6d8dd2736 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.725510 May 28 03:36:04.703364 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 21/34] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:03 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:04.767704 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7df6f491-90c2-49fa-a9ab-b1cb5167addb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:04.812016 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7df6f491-90c2-49fa-a9ab-b1cb5167addb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:04.812016 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7df6f491-90c2-49fa-a9ab-b1cb5167addb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:04.813630 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7df6f491-90c2-49fa-a9ab-b1cb5167addb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1282 microversion: 2.20 time: 0.292696 May 28 03:36:04.814988 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 14/35] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:04 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1282 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:36:05.696459 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-87425301-6a15-48d9-b4a5-47c1083a77ca tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:05.697727 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-87425301-6a15-48d9-b4a5-47c1083a77ca tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:05.710340 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-87425301-6a15-48d9-b4a5-47c1083a77ca tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:05.712027 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-87425301-6a15-48d9-b4a5-47c1083a77ca tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:05.713002 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-87425301-6a15-48d9-b4a5-47c1083a77ca tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:05.716014 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-87425301-6a15-48d9-b4a5-47c1083a77ca tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:05.832370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f8c8d8d8-4a08-4dce-9ea5-cbe9dc23b3db tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:05.834092 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f8c8d8d8-4a08-4dce-9ea5-cbe9dc23b3db tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:05.847453 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f8c8d8d8-4a08-4dce-9ea5-cbe9dc23b3db tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:05.849652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f8c8d8d8-4a08-4dce-9ea5-cbe9dc23b3db tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:05.852721 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f8c8d8d8-4a08-4dce-9ea5-cbe9dc23b3db tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:05.854131 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f8c8d8d8-4a08-4dce-9ea5-cbe9dc23b3db tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:06.196246 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f8c8d8d8-4a08-4dce-9ea5-cbe9dc23b3db tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:06.217092 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f8c8d8d8-4a08-4dce-9ea5-cbe9dc23b3db tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:06.218938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f8c8d8d8-4a08-4dce-9ea5-cbe9dc23b3db tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:06.225541 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f8c8d8d8-4a08-4dce-9ea5-cbe9dc23b3db tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1286 microversion: 2.20 time: 0.397228 May 28 03:36:06.246946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 15/36] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:05 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1286 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:07.144650 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-87425301-6a15-48d9-b4a5-47c1083a77ca tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:07.158729 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-87425301-6a15-48d9-b4a5-47c1083a77ca tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 1.461968 May 28 03:36:07.159422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 22/37] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:05 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 1468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:07.264883 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8d675db7-9c3c-4ba0-aa1a-7cafe58ec1ee tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:07.265563 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8d675db7-9c3c-4ba0-aa1a-7cafe58ec1ee tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:07.306768 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8d675db7-9c3c-4ba0-aa1a-7cafe58ec1ee tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:07.307228 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8d675db7-9c3c-4ba0-aa1a-7cafe58ec1ee tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:07.308611 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8d675db7-9c3c-4ba0-aa1a-7cafe58ec1ee tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:07.309291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8d675db7-9c3c-4ba0-aa1a-7cafe58ec1ee tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:08.038214 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8d675db7-9c3c-4ba0-aa1a-7cafe58ec1ee tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:08.068500 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8d675db7-9c3c-4ba0-aa1a-7cafe58ec1ee tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:08.077029 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8d675db7-9c3c-4ba0-aa1a-7cafe58ec1ee tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:08.079780 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8d675db7-9c3c-4ba0-aa1a-7cafe58ec1ee tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1334 microversion: 2.20 time: 0.830038 May 28 03:36:08.095440 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 16/38] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:07 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1334 bytes in 846 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:08.176453 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4092ac16-68bd-47c0-bb14-85bdda724879 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:08.177057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4092ac16-68bd-47c0-bb14-85bdda724879 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:08.204736 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4092ac16-68bd-47c0-bb14-85bdda724879 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:08.205819 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4092ac16-68bd-47c0-bb14-85bdda724879 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:08.207476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4092ac16-68bd-47c0-bb14-85bdda724879 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:08.211273 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4092ac16-68bd-47c0-bb14-85bdda724879 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:09.006499 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4092ac16-68bd-47c0-bb14-85bdda724879 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:09.011070 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1656168052", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:09.024184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4092ac16-68bd-47c0-bb14-85bdda724879 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.851476 May 28 03:36:09.025577 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 23/39] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:08 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 853 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:09.096834 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.network.neutron [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] validate_networks() for [('a3076c7d-656d-4e5b-8611-4fd5569b4e24', None, None, None, None, None)] {{(pid=124605) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:36:09.109413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9efaa6e2-4aec-4ece-8522-79271a3f2326 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:09.110761 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9efaa6e2-4aec-4ece-8522-79271a3f2326 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:09.119868 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9efaa6e2-4aec-4ece-8522-79271a3f2326 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:09.120539 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9efaa6e2-4aec-4ece-8522-79271a3f2326 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:09.121872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9efaa6e2-4aec-4ece-8522-79271a3f2326 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:09.122487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9efaa6e2-4aec-4ece-8522-79271a3f2326 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:09.323220 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9efaa6e2-4aec-4ece-8522-79271a3f2326 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:09.335329 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9efaa6e2-4aec-4ece-8522-79271a3f2326 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:09.337202 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9efaa6e2-4aec-4ece-8522-79271a3f2326 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:09.338634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9efaa6e2-4aec-4ece-8522-79271a3f2326 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1340 microversion: 2.20 time: 0.238430 May 28 03:36:09.340238 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 24/40] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:09 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1340 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:09.347270 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-172742705", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ee7025a6-8632-48bc-9a4d-e0b6d8e98a79"}], "security_groups": [{"name": "tempest-securitygroup--975371077"}], "key_name": "tempest-keypair-2087622025", "user_data": "}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:09.726987 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Flavor limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:36:09.727427 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Image limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:36:09.727910 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Flavor pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:36:09.728400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Image pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:36:09.728852 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:36:09.744399 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:09.761019 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] validate_networks() for [('ee7025a6-8632-48bc-9a4d-e0b6d8e98a79', None, None, None, None, None)] {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:36:09.761134 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:09.761134 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:09.761519 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:09.827631 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Getting quotas for project 5cab8255c6134aebb9c93493a2342cb1. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:36:09.842498 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Getting quotas for user b6709eed72ca46468fd298929ae0f764 and project 5cab8255c6134aebb9c93493a2342cb1. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:36:09.882565 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Going to run 1 instances... {{(pid=124605) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:36:09.886971 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:36:10.108202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-06faee18-cb2e-4531-98bf-4d14b6754587 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.360237 May 28 03:36:10.113419 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 17/41] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:36:08 2022] POST /compute/v2.1/servers => generated 383 bytes in 1365 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:36:10.127294 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ee469c3c-307d-4639-be8c-1e443e836ac7 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:10.128163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ee469c3c-307d-4639-be8c-1e443e836ac7 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:10.143068 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ee469c3c-307d-4639-be8c-1e443e836ac7 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:10.144101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ee469c3c-307d-4639-be8c-1e443e836ac7 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:10.145134 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ee469c3c-307d-4639-be8c-1e443e836ac7 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:10.145742 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ee469c3c-307d-4639-be8c-1e443e836ac7 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:10.519218 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ee469c3c-307d-4639-be8c-1e443e836ac7 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:10.532107 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ee469c3c-307d-4639-be8c-1e443e836ac7 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.414535 May 28 03:36:10.535664 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 18/42] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:10 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:10.562082 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1dd5f71f-c5a3-4914-929f-740496733078 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:10.562082 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1dd5f71f-c5a3-4914-929f-740496733078 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:10.803779 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1dd5f71f-c5a3-4914-929f-740496733078 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:10.817591 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1dd5f71f-c5a3-4914-929f-740496733078 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1222 microversion: 2.1 time: 0.278626 May 28 03:36:10.818371 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 19/43] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:10 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1222 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:10.838893 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0793ff0a-23eb-4572-894c-5d3e1deed930 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:10.854993 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0793ff0a-23eb-4572-894c-5d3e1deed930 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:10.863845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=11 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:10.868155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0793ff0a-23eb-4572-894c-5d3e1deed930 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:10.868908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0793ff0a-23eb-4572-894c-5d3e1deed930 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:10.870703 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0793ff0a-23eb-4572-894c-5d3e1deed930 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:10.872344 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0793ff0a-23eb-4572-894c-5d3e1deed930 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:11.204702 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0793ff0a-23eb-4572-894c-5d3e1deed930 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:11.215703 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0793ff0a-23eb-4572-894c-5d3e1deed930 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:11.223780 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0793ff0a-23eb-4572-894c-5d3e1deed930 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:11.236036 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0793ff0a-23eb-4572-894c-5d3e1deed930 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1340 microversion: 2.20 time: 0.414727 May 28 03:36:11.247094 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 20/44] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:10 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1340 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:11.261926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1915859703", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:11.396287 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.network.neutron [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] validate_networks() for [('a3076c7d-656d-4e5b-8611-4fd5569b4e24', None, None, None, None, None)] {{(pid=124605) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:36:13.151463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:36:13.152003 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:36:13.152708 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:36:13.153259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:36:13.153790 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:36:13.200702 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:13.201649 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:13.207078 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:13.208070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:13.323640 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:36:13.331278 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for user 9d420e4b554e4deab21b89094e0547ca and project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:36:13.371016 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:36:13.374041 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:36:13.635097 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-7ea6f7a7-3679-4ea4-805f-63670f3416e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 4.291828 May 28 03:36:13.637551 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 25/45] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:36:09 2022] POST /compute/v2.1/servers => generated 401 bytes in 4296 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:36:13.654004 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1017ef63-b34b-4989-9db6-f92f8c3ea808 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:13.655761 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1017ef63-b34b-4989-9db6-f92f8c3ea808 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:13.700994 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1017ef63-b34b-4989-9db6-f92f8c3ea808 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:13.702227 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1017ef63-b34b-4989-9db6-f92f8c3ea808 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:13.706856 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1017ef63-b34b-4989-9db6-f92f8c3ea808 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:13.710135 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1017ef63-b34b-4989-9db6-f92f8c3ea808 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:14.717642 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Flavor limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:36:14.718796 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Image limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:36:14.719461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Flavor pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:36:14.720067 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Image pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:36:14.720695 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:36:14.765141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:14.768089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:14.771266 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:14.774885 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:14.831868 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Getting quotas for project 5cab8255c6134aebb9c93493a2342cb1. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:36:14.841852 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Getting quotas for user b6709eed72ca46468fd298929ae0f764 and project 5cab8255c6134aebb9c93493a2342cb1. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:36:14.865950 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Going to run 1 instances... {{(pid=124605) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:36:14.889719 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:36:15.040520 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1017ef63-b34b-4989-9db6-f92f8c3ea808 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:15.054216 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1017ef63-b34b-4989-9db6-f92f8c3ea808 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 1.413663 May 28 03:36:15.064048 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 26/46] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:13 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 1426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:15.172091 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-774cd9f9-5e21-4bd8-b4a9-277645b9912f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.922235 May 28 03:36:15.175846 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 21/47] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:36:11 2022] POST /compute/v2.1/servers => generated 383 bytes in 3928 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:36:15.200000 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-500b9ad7-8172-4f27-ab37-f4c06b4ae430 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:15.206847 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-500b9ad7-8172-4f27-ab37-f4c06b4ae430 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:15.239769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-500b9ad7-8172-4f27-ab37-f4c06b4ae430 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:15.240368 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-500b9ad7-8172-4f27-ab37-f4c06b4ae430 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:15.241347 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-500b9ad7-8172-4f27-ab37-f4c06b4ae430 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:15.241912 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-500b9ad7-8172-4f27-ab37-f4c06b4ae430 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:15.594192 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-3af399c8-f17b-4f92-ba34-bee850d32305 req-b1e849c2-9a33-4115-ac6c-47bf394d00d2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "362687d7-ef88-4e49-b7a5-d732c28e1b0b", "tag": "cda792d3-ee73-4947-a077-2caf88494a15"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:15.601482 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=4 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:15.630105 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-3af399c8-f17b-4f92-ba34-bee850d32305 req-b1e849c2-9a33-4115-ac6c-47bf394d00d2 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:15.635308 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-3af399c8-f17b-4f92-ba34-bee850d32305 req-b1e849c2-9a33-4115-ac6c-47bf394d00d2 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:15.692866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-3af399c8-f17b-4f92-ba34-bee850d32305 req-b1e849c2-9a33-4115-ac6c-47bf394d00d2 service nova] Creating event network-changed:cda792d3-ee73-4947-a077-2caf88494a15 for instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b on hv-cinder-84336 May 28 03:36:15.694832 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-3af399c8-f17b-4f92-ba34-bee850d32305 req-b1e849c2-9a33-4115-ac6c-47bf394d00d2 service nova] Acquired lock "compute-rpcapi-router" {{(pid=124604) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 28 03:36:15.697126 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-72f16ac6-dd40-4e18-ad2f-fb5e2f69f316 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:15.699905 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-72f16ac6-dd40-4e18-ad2f-fb5e2f69f316 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:15.700949 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-72f16ac6-dd40-4e18-ad2f-fb5e2f69f316 None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:15.704049 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-72f16ac6-dd40-4e18-ad2f-fb5e2f69f316 None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:15.730693 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.compute.rpcapi [None req-72f16ac6-dd40-4e18-ad2f-fb5e2f69f316 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 28 03:36:15.731978 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-72f16ac6-dd40-4e18-ad2f-fb5e2f69f316 None None] Releasing lock "compute-rpcapi-router" {{(pid=124604) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 28 03:36:15.791083 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-72f16ac6-dd40-4e18-ad2f-fb5e2f69f316 None None] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.725008 May 28 03:36:15.792146 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 27/48] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 727 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:15.803152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-41e2ebb8-12aa-4a35-97c9-10f78fea19cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:15.804242 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-41e2ebb8-12aa-4a35-97c9-10f78fea19cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:15.820253 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-41e2ebb8-12aa-4a35-97c9-10f78fea19cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:15.823225 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-41e2ebb8-12aa-4a35-97c9-10f78fea19cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:15.827458 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-41e2ebb8-12aa-4a35-97c9-10f78fea19cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:15.828011 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-41e2ebb8-12aa-4a35-97c9-10f78fea19cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:16.131078 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-500b9ad7-8172-4f27-ab37-f4c06b4ae430 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:16.144232 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-500b9ad7-8172-4f27-ab37-f4c06b4ae430 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:16.160685 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-500b9ad7-8172-4f27-ab37-f4c06b4ae430 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:16.168893 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-500b9ad7-8172-4f27-ab37-f4c06b4ae430 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.986753 May 28 03:36:16.170577 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 22/49] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:15 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 994 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:16.184177 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1101351884", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "9cd091c1-088f-4b49-b9d5-c93354a4ddb5"}], "security_groups": [{"name": "tempest-securitygroup--1971261680"}], "key_name": "tempest-keypair-447830529", "user_data": "}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:16.509501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-41e2ebb8-12aa-4a35-97c9-10f78fea19cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:16.528131 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-41e2ebb8-12aa-4a35-97c9-10f78fea19cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1245 microversion: 2.1 time: 0.734343 May 28 03:36:16.532319 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 28/50] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:15 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1245 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:16.549773 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d1216529-ab99-47a2-94a2-277c8550efff tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:16.553683 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d1216529-ab99-47a2-94a2-277c8550efff tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:16.556653 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.network.neutron [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] validate_networks() for [('9cd091c1-088f-4b49-b9d5-c93354a4ddb5', None, None, None, None, None)] {{(pid=124605) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:36:16.648452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d1216529-ab99-47a2-94a2-277c8550efff tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:16.661413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=17 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:16.677453 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d1216529-ab99-47a2-94a2-277c8550efff tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1222 microversion: 2.1 time: 0.138589 May 28 03:36:16.678641 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 29/51] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:16 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1222 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:16.695921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8d2efeef-efb7-42ad-95b1-28ffb3c7e7de tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:16.696738 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8d2efeef-efb7-42ad-95b1-28ffb3c7e7de tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:16.710770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d2efeef-efb7-42ad-95b1-28ffb3c7e7de tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:16.725647 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d2efeef-efb7-42ad-95b1-28ffb3c7e7de tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:16.726962 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d2efeef-efb7-42ad-95b1-28ffb3c7e7de tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:16.732503 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d2efeef-efb7-42ad-95b1-28ffb3c7e7de tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:17.052114 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8d2efeef-efb7-42ad-95b1-28ffb3c7e7de tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:17.061897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8d2efeef-efb7-42ad-95b1-28ffb3c7e7de tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.380768 May 28 03:36:17.062577 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 30/52] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:16 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:17.187893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-87daf286-104b-47e1-8078-099080391cbf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:17.188601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-87daf286-104b-47e1-8078-099080391cbf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:17.198135 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-87daf286-104b-47e1-8078-099080391cbf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:17.198687 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-87daf286-104b-47e1-8078-099080391cbf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:17.199694 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-87daf286-104b-47e1-8078-099080391cbf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:17.200351 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-87daf286-104b-47e1-8078-099080391cbf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:17.397968 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-87daf286-104b-47e1-8078-099080391cbf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:17.414211 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-87daf286-104b-47e1-8078-099080391cbf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:17.416056 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-87daf286-104b-47e1-8078-099080391cbf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:17.417462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-87daf286-104b-47e1-8078-099080391cbf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.233381 May 28 03:36:17.418516 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 31/53] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:17 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:36:17.552604 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-aeacf743-392d-48e8-a4a0-9c1cd081375a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:17.552604 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-aeacf743-392d-48e8-a4a0-9c1cd081375a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:17.571241 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Flavor limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:36:17.572954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Image limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:36:17.573619 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Flavor pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:36:17.575505 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-aeacf743-392d-48e8-a4a0-9c1cd081375a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:17.576988 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Image pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:36:17.577530 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:36:17.581536 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-aeacf743-392d-48e8-a4a0-9c1cd081375a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:17.582121 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-aeacf743-392d-48e8-a4a0-9c1cd081375a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:17.582646 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-aeacf743-392d-48e8-a4a0-9c1cd081375a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:17.599414 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:17.600911 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:17.602904 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:17.606348 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:17.651462 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Getting quotas for project 70eaf602c8db4ec1bfb26e5eb3b13c82. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:36:17.662385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Getting quotas for user dc6a3335562448d1934fa18e8f51b119 and project 70eaf602c8db4ec1bfb26e5eb3b13c82. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:36:17.703918 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Going to run 1 instances... {{(pid=124605) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:36:17.709462 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:36:17.749949 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-aeacf743-392d-48e8-a4a0-9c1cd081375a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:17.762071 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-aeacf743-392d-48e8-a4a0-9c1cd081375a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1245 microversion: 2.1 time: 0.217227 May 28 03:36:17.763270 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 32/54] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:17 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1245 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:17.769373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4aeca33c-cf22-4b66-a850-2539aa66fceb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:17.770236 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4aeca33c-cf22-4b66-a850-2539aa66fceb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:17.776584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4aeca33c-cf22-4b66-a850-2539aa66fceb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:17.779733 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4aeca33c-cf22-4b66-a850-2539aa66fceb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:17.779733 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4aeca33c-cf22-4b66-a850-2539aa66fceb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:17.779733 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4aeca33c-cf22-4b66-a850-2539aa66fceb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:17.849921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4aeca33c-cf22-4b66-a850-2539aa66fceb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:17.859001 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4aeca33c-cf22-4b66-a850-2539aa66fceb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1226 microversion: 2.1 time: 0.094440 May 28 03:36:17.868825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 33/55] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:17 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:17.880820 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-69773af0-3984-48b1-a8ae-99e1b7b45c12 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.708861 May 28 03:36:17.881503 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 23/56] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:36:16 2022] POST /compute/v2.1/servers => generated 402 bytes in 1710 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:36:17.892085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b1580795-04fe-4215-ae00-3c1bed11bc54 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:17.892775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b1580795-04fe-4215-ae00-3c1bed11bc54 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:18.018445 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b1580795-04fe-4215-ae00-3c1bed11bc54 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:18.027148 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b1580795-04fe-4215-ae00-3c1bed11bc54 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1237 microversion: 2.1 time: 0.139420 May 28 03:36:18.028023 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 34/57] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:17 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1237 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:36:18.079814 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-229c95cd-c444-427f-a61c-e83dd26e6af9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:18.080715 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-229c95cd-c444-427f-a61c-e83dd26e6af9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:18.105326 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-229c95cd-c444-427f-a61c-e83dd26e6af9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:18.106073 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-229c95cd-c444-427f-a61c-e83dd26e6af9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:18.109260 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-229c95cd-c444-427f-a61c-e83dd26e6af9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:18.115441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-229c95cd-c444-427f-a61c-e83dd26e6af9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:18.437577 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a2c7e5a8-784c-493b-abf1-8def698b31d4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:18.438418 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a2c7e5a8-784c-493b-abf1-8def698b31d4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:18.455317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2c7e5a8-784c-493b-abf1-8def698b31d4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:18.459235 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2c7e5a8-784c-493b-abf1-8def698b31d4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:18.466014 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2c7e5a8-784c-493b-abf1-8def698b31d4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:18.470014 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2c7e5a8-784c-493b-abf1-8def698b31d4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:19.075399 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-229c95cd-c444-427f-a61c-e83dd26e6af9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:19.085146 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-229c95cd-c444-427f-a61c-e83dd26e6af9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 1.009248 May 28 03:36:19.087255 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 24/58] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:18 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 1012 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:19.104497 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a3c3e07c-67a0-4a58-9025-0f0196f35281 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:19.109449 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a3c3e07c-67a0-4a58-9025-0f0196f35281 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:19.135493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3c3e07c-67a0-4a58-9025-0f0196f35281 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:19.135493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3c3e07c-67a0-4a58-9025-0f0196f35281 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:19.135493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3c3e07c-67a0-4a58-9025-0f0196f35281 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:19.135493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3c3e07c-67a0-4a58-9025-0f0196f35281 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:19.225970 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a2c7e5a8-784c-493b-abf1-8def698b31d4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:19.238183 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a2c7e5a8-784c-493b-abf1-8def698b31d4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:19.266877 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a2c7e5a8-784c-493b-abf1-8def698b31d4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:19.266877 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a2c7e5a8-784c-493b-abf1-8def698b31d4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.808129 May 28 03:36:19.266877 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 35/59] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:18 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 817 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:19.266877 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2be6203d-4920-4248-b6bb-bb2014ecc0fc tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:19.268016 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2be6203d-4920-4248-b6bb-bb2014ecc0fc tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:19.271872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2be6203d-4920-4248-b6bb-bb2014ecc0fc tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:19.272323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2be6203d-4920-4248-b6bb-bb2014ecc0fc tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:19.273146 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2be6203d-4920-4248-b6bb-bb2014ecc0fc tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:19.273581 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2be6203d-4920-4248-b6bb-bb2014ecc0fc tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:19.300385 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:19.370607 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a3c3e07c-67a0-4a58-9025-0f0196f35281 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:19.372778 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=57 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:19.380332 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a3c3e07c-67a0-4a58-9025-0f0196f35281 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1293 microversion: 2.1 time: 0.291744 May 28 03:36:19.380931 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 25/60] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:19 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1293 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:19.530807 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-2be6203d-4920-4248-b6bb-bb2014ecc0fc tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:19.546401 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2be6203d-4920-4248-b6bb-bb2014ecc0fc tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1226 microversion: 2.1 time: 0.295992 May 28 03:36:19.556917 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 36/61] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:19 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1226 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:19.559881 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-7e91d32c-559e-4b5b-b349-94345fb0aa2f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:19.560695 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-7e91d32c-559e-4b5b-b349-94345fb0aa2f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:19.673340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-5c6238f6-e2af-4ebb-9c6c-96c3ab59ae93 req-cc21d7c3-6fcc-4255-a5fe-3978cf0695ce service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35a9c645-590f-49e1-ab03-af4167f10547", "tag": "97df7472-c1e5-4bbd-80b5-6efb1965aeb9"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:19.673340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-5c6238f6-e2af-4ebb-9c6c-96c3ab59ae93 req-cc21d7c3-6fcc-4255-a5fe-3978cf0695ce service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:19.673340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-5c6238f6-e2af-4ebb-9c6c-96c3ab59ae93 req-cc21d7c3-6fcc-4255-a5fe-3978cf0695ce service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:19.693368 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-5c6238f6-e2af-4ebb-9c6c-96c3ab59ae93 req-cc21d7c3-6fcc-4255-a5fe-3978cf0695ce service nova] Creating event network-changed:97df7472-c1e5-4bbd-80b5-6efb1965aeb9 for instance 35a9c645-590f-49e1-ab03-af4167f10547 on hv-cinder-84336 May 28 03:36:19.693905 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-5c6238f6-e2af-4ebb-9c6c-96c3ab59ae93 req-cc21d7c3-6fcc-4255-a5fe-3978cf0695ce service nova] Acquired lock "compute-rpcapi-router" {{(pid=124605) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 28 03:36:19.695007 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78166610-fd40-46f4-aa9a-45557af90714 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:19.695556 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78166610-fd40-46f4-aa9a-45557af90714 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:19.696330 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78166610-fd40-46f4-aa9a-45557af90714 None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:19.696755 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78166610-fd40-46f4-aa9a-45557af90714 None None] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:19.702803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=11 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:19.704598 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-7e91d32c-559e-4b5b-b349-94345fb0aa2f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:19.711017 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.compute.rpcapi [None req-78166610-fd40-46f4-aa9a-45557af90714 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 28 03:36:19.712206 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78166610-fd40-46f4-aa9a-45557af90714 None None] Releasing lock "compute-rpcapi-router" {{(pid=124605) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 28 03:36:19.714397 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-7e91d32c-559e-4b5b-b349-94345fb0aa2f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1237 microversion: 2.1 time: 0.161699 May 28 03:36:19.715232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 37/62] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:19 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1237 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:19.730629 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-78166610-fd40-46f4-aa9a-45557af90714 None None] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.348419 May 28 03:36:19.731345 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 26/63] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:20.105456 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d5957a80-0e13-4894-9989-b84a1eeeb060 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:20.106163 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d5957a80-0e13-4894-9989-b84a1eeeb060 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:20.112573 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5957a80-0e13-4894-9989-b84a1eeeb060 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:20.116911 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5957a80-0e13-4894-9989-b84a1eeeb060 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:20.147510 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5957a80-0e13-4894-9989-b84a1eeeb060 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:20.152565 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5957a80-0e13-4894-9989-b84a1eeeb060 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:20.288143 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-49783003-10fa-4fc9-8b69-49287666bc08 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:20.289948 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-49783003-10fa-4fc9-8b69-49287666bc08 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:20.299110 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-49783003-10fa-4fc9-8b69-49287666bc08 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:20.299820 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-49783003-10fa-4fc9-8b69-49287666bc08 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:20.312688 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-49783003-10fa-4fc9-8b69-49287666bc08 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:20.312688 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-49783003-10fa-4fc9-8b69-49287666bc08 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:20.652325 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-49783003-10fa-4fc9-8b69-49287666bc08 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:20.658489 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-49783003-10fa-4fc9-8b69-49287666bc08 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:20.660122 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-49783003-10fa-4fc9-8b69-49287666bc08 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:20.661344 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-49783003-10fa-4fc9-8b69-49287666bc08 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.379529 May 28 03:36:20.661585 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d5957a80-0e13-4894-9989-b84a1eeeb060 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:20.671244 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d5957a80-0e13-4894-9989-b84a1eeeb060 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.569704 May 28 03:36:20.671807 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 38/64] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:20 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:20.672993 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 27/65] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:20 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:20.678261 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-74ed7197-0dcc-48ca-aaa9-c213e3681984 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:20.678871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-74ed7197-0dcc-48ca-aaa9-c213e3681984 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:20.683978 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-28945b39-0f51-4f07-b8f1-e37b2544b883 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:20.685941 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-28945b39-0f51-4f07-b8f1-e37b2544b883 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:20.686047 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-74ed7197-0dcc-48ca-aaa9-c213e3681984 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:20.686047 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-74ed7197-0dcc-48ca-aaa9-c213e3681984 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:20.686661 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-74ed7197-0dcc-48ca-aaa9-c213e3681984 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:20.687106 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-74ed7197-0dcc-48ca-aaa9-c213e3681984 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:20.692971 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-28945b39-0f51-4f07-b8f1-e37b2544b883 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:20.692971 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-28945b39-0f51-4f07-b8f1-e37b2544b883 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:20.692971 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-28945b39-0f51-4f07-b8f1-e37b2544b883 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:20.693403 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-28945b39-0f51-4f07-b8f1-e37b2544b883 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:20.754661 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-74ed7197-0dcc-48ca-aaa9-c213e3681984 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:20.763161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-74ed7197-0dcc-48ca-aaa9-c213e3681984 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1274 microversion: 2.1 time: 0.088956 May 28 03:36:20.764120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 28/66] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:20 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1274 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:20.769030 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d65a1de1-5f2f-49c9-b8b9-d76f26db545b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:20.779842 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d65a1de1-5f2f-49c9-b8b9-d76f26db545b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:20.784613 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-28945b39-0f51-4f07-b8f1-e37b2544b883 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:20.795529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-28945b39-0f51-4f07-b8f1-e37b2544b883 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1301 microversion: 2.1 time: 0.122681 May 28 03:36:20.796129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 39/67] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:20 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1301 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:20.842789 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d65a1de1-5f2f-49c9-b8b9-d76f26db545b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:20.842789 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=47 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:20.847343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d65a1de1-5f2f-49c9-b8b9-d76f26db545b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1237 microversion: 2.1 time: 0.082265 May 28 03:36:20.847919 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 29/68] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:20 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1237 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:20.965012 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-53a715af-e353-4fb8-8d7b-ec6788e42c4c req-5ea29663-ad88-4126-9ec8-2c9527d24ea1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3b4a0f56-eb59-4f13-a90a-442b9a2e0af8", "tag": "2038175b-b88a-437c-9114-63208c464911"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:20.974441 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-53a715af-e353-4fb8-8d7b-ec6788e42c4c req-5ea29663-ad88-4126-9ec8-2c9527d24ea1 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:20.974987 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-53a715af-e353-4fb8-8d7b-ec6788e42c4c req-5ea29663-ad88-4126-9ec8-2c9527d24ea1 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:20.988001 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-53a715af-e353-4fb8-8d7b-ec6788e42c4c req-5ea29663-ad88-4126-9ec8-2c9527d24ea1 service nova] Creating event network-changed:2038175b-b88a-437c-9114-63208c464911 for instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 on hv-cinder-84336 May 28 03:36:20.993134 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-53a715af-e353-4fb8-8d7b-ec6788e42c4c req-5ea29663-ad88-4126-9ec8-2c9527d24ea1 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032483 May 28 03:36:20.993795 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 40/69] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:36:21.680109 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a9711527-7261-4090-82bb-6b5667442133 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:21.680712 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a9711527-7261-4090-82bb-6b5667442133 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:21.688201 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a9711527-7261-4090-82bb-6b5667442133 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:21.688656 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a9711527-7261-4090-82bb-6b5667442133 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:21.689466 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a9711527-7261-4090-82bb-6b5667442133 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:21.695837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-35be6454-0f39-4d75-be9a-c945cb0f8095 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:21.695837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-35be6454-0f39-4d75-be9a-c945cb0f8095 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:21.696356 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a9711527-7261-4090-82bb-6b5667442133 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:21.700764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-35be6454-0f39-4d75-be9a-c945cb0f8095 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:21.701375 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-35be6454-0f39-4d75-be9a-c945cb0f8095 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:21.702159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-35be6454-0f39-4d75-be9a-c945cb0f8095 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:21.702764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-35be6454-0f39-4d75-be9a-c945cb0f8095 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:21.864472 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-35be6454-0f39-4d75-be9a-c945cb0f8095 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:21.871375 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a9711527-7261-4090-82bb-6b5667442133 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:21.872679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-35be6454-0f39-4d75-be9a-c945cb0f8095 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.186079 May 28 03:36:21.873297 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 41/70] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:21 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:36:21.878266 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f110b573-fbb8-4618-81b3-334888969bfb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:21.878883 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f110b573-fbb8-4618-81b3-334888969bfb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:21.883448 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a9711527-7261-4090-82bb-6b5667442133 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:21.884633 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f110b573-fbb8-4618-81b3-334888969bfb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:21.884988 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a9711527-7261-4090-82bb-6b5667442133 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:21.885091 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f110b573-fbb8-4618-81b3-334888969bfb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:21.885879 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f110b573-fbb8-4618-81b3-334888969bfb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:21.886226 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a9711527-7261-4090-82bb-6b5667442133 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.210902 May 28 03:36:21.886348 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f110b573-fbb8-4618-81b3-334888969bfb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:21.886827 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 30/71] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:21 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:21.891157 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9aaf1244-9b66-4f4d-85dd-c5b3ac7c00e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:21.891764 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9aaf1244-9b66-4f4d-85dd-c5b3ac7c00e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:21.898062 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9aaf1244-9b66-4f4d-85dd-c5b3ac7c00e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:21.898507 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9aaf1244-9b66-4f4d-85dd-c5b3ac7c00e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:21.899266 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9aaf1244-9b66-4f4d-85dd-c5b3ac7c00e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:21.899698 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9aaf1244-9b66-4f4d-85dd-c5b3ac7c00e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:21.982536 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9aaf1244-9b66-4f4d-85dd-c5b3ac7c00e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:21.993236 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9aaf1244-9b66-4f4d-85dd-c5b3ac7c00e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1301 microversion: 2.1 time: 0.105297 May 28 03:36:21.993730 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 31/72] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:21 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1301 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:22.000047 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2db6b9af-c703-43f4-b5df-20771570fa21 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:22.000771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2db6b9af-c703-43f4-b5df-20771570fa21 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:22.003236 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f110b573-fbb8-4618-81b3-334888969bfb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:22.009348 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2db6b9af-c703-43f4-b5df-20771570fa21 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:22.009967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2db6b9af-c703-43f4-b5df-20771570fa21 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:22.010705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2db6b9af-c703-43f4-b5df-20771570fa21 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:22.011166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f110b573-fbb8-4618-81b3-334888969bfb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1274 microversion: 2.1 time: 0.136891 May 28 03:36:22.011324 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2db6b9af-c703-43f4-b5df-20771570fa21 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:22.011761 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 42/73] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:21 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1274 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:22.092685 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2db6b9af-c703-43f4-b5df-20771570fa21 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:22.100672 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2db6b9af-c703-43f4-b5df-20771570fa21 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1241 microversion: 2.1 time: 0.105707 May 28 03:36:22.101459 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 32/74] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:21 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1241 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:22.890378 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3646d84f-56ae-41f8-9300-a36211922f49 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:22.891094 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3646d84f-56ae-41f8-9300-a36211922f49 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:22.897619 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3646d84f-56ae-41f8-9300-a36211922f49 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:22.898167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3646d84f-56ae-41f8-9300-a36211922f49 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:22.898941 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3646d84f-56ae-41f8-9300-a36211922f49 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:22.899438 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3646d84f-56ae-41f8-9300-a36211922f49 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:22.905357 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-823dfdab-3add-48dd-8694-4e4af846acca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:22.908221 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-823dfdab-3add-48dd-8694-4e4af846acca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:22.915710 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-823dfdab-3add-48dd-8694-4e4af846acca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:22.916576 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-823dfdab-3add-48dd-8694-4e4af846acca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:22.917695 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-823dfdab-3add-48dd-8694-4e4af846acca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:22.918335 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-823dfdab-3add-48dd-8694-4e4af846acca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:23.043860 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3646d84f-56ae-41f8-9300-a36211922f49 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:23.053218 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3646d84f-56ae-41f8-9300-a36211922f49 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.166442 May 28 03:36:23.062916 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 43/75] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:22 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:23.068139 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5615e514-20fa-411b-a5da-06dfbbed624c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:23.069810 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5615e514-20fa-411b-a5da-06dfbbed624c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:23.077143 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5615e514-20fa-411b-a5da-06dfbbed624c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:23.080698 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5615e514-20fa-411b-a5da-06dfbbed624c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:23.080698 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5615e514-20fa-411b-a5da-06dfbbed624c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:23.080698 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5615e514-20fa-411b-a5da-06dfbbed624c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:23.088173 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-823dfdab-3add-48dd-8694-4e4af846acca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:23.100348 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-823dfdab-3add-48dd-8694-4e4af846acca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:23.102579 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-823dfdab-3add-48dd-8694-4e4af846acca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:23.103880 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-823dfdab-3add-48dd-8694-4e4af846acca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.203259 May 28 03:36:23.105244 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 33/76] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:22 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:23.111617 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-96de2985-6c0e-42df-b361-8b1a6d664237 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:23.112261 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-96de2985-6c0e-42df-b361-8b1a6d664237 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:23.118641 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-96de2985-6c0e-42df-b361-8b1a6d664237 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:23.120871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-96de2985-6c0e-42df-b361-8b1a6d664237 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:23.120871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-96de2985-6c0e-42df-b361-8b1a6d664237 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:23.120871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-96de2985-6c0e-42df-b361-8b1a6d664237 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:23.166356 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5615e514-20fa-411b-a5da-06dfbbed624c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:23.174801 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5615e514-20fa-411b-a5da-06dfbbed624c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1301 microversion: 2.1 time: 0.111194 May 28 03:36:23.176102 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 44/77] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:23 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1301 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:23.192271 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-046fae15-8402-4f18-85fd-7b9e366aa6ef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:23.193624 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-046fae15-8402-4f18-85fd-7b9e366aa6ef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:23.195640 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-96de2985-6c0e-42df-b361-8b1a6d664237 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:23.202655 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-046fae15-8402-4f18-85fd-7b9e366aa6ef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:23.203107 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-046fae15-8402-4f18-85fd-7b9e366aa6ef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:23.203762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-96de2985-6c0e-42df-b361-8b1a6d664237 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1274 microversion: 2.1 time: 0.097392 May 28 03:36:23.203873 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-046fae15-8402-4f18-85fd-7b9e366aa6ef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:23.204316 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-046fae15-8402-4f18-85fd-7b9e366aa6ef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:23.206706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 34/78] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:23 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1274 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:23.278759 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-046fae15-8402-4f18-85fd-7b9e366aa6ef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:23.286288 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-046fae15-8402-4f18-85fd-7b9e366aa6ef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1241 microversion: 2.1 time: 0.108948 May 28 03:36:23.286965 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 45/79] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:23 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1241 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:24.071283 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f583c8e0-9c67-4cc9-90bd-525645fa3106 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:24.071894 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f583c8e0-9c67-4cc9-90bd-525645fa3106 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:24.078309 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f583c8e0-9c67-4cc9-90bd-525645fa3106 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:24.078760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f583c8e0-9c67-4cc9-90bd-525645fa3106 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:24.079521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f583c8e0-9c67-4cc9-90bd-525645fa3106 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:24.079962 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f583c8e0-9c67-4cc9-90bd-525645fa3106 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:24.124351 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d9b53324-3ae3-4f8b-ad91-6f0bffee22e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:24.125702 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d9b53324-3ae3-4f8b-ad91-6f0bffee22e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:24.132494 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9b53324-3ae3-4f8b-ad91-6f0bffee22e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:24.132933 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9b53324-3ae3-4f8b-ad91-6f0bffee22e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:24.133778 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9b53324-3ae3-4f8b-ad91-6f0bffee22e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:24.134251 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9b53324-3ae3-4f8b-ad91-6f0bffee22e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:24.289813 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d9b53324-3ae3-4f8b-ad91-6f0bffee22e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:24.302817 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d9b53324-3ae3-4f8b-ad91-6f0bffee22e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:24.310763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f583c8e0-9c67-4cc9-90bd-525645fa3106 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:24.317378 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d9b53324-3ae3-4f8b-ad91-6f0bffee22e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:24.318791 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d9b53324-3ae3-4f8b-ad91-6f0bffee22e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.198754 May 28 03:36:24.319458 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 46/80] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:24 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:24.326868 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f583c8e0-9c67-4cc9-90bd-525645fa3106 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.259294 May 28 03:36:24.328069 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 35/81] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:24 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:24.331556 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-07b31487-6189-4892-aed4-497f16d96a9e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:24.334895 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-07b31487-6189-4892-aed4-497f16d96a9e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:24.342725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-79f831c1-e1b0-4e92-8e18-2bb9fa5155e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:24.344504 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-79f831c1-e1b0-4e92-8e18-2bb9fa5155e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:24.347204 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07b31487-6189-4892-aed4-497f16d96a9e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:24.347657 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07b31487-6189-4892-aed4-497f16d96a9e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:24.348436 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07b31487-6189-4892-aed4-497f16d96a9e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:24.348876 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07b31487-6189-4892-aed4-497f16d96a9e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:24.354918 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-79f831c1-e1b0-4e92-8e18-2bb9fa5155e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:24.355472 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-79f831c1-e1b0-4e92-8e18-2bb9fa5155e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:24.356311 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-79f831c1-e1b0-4e92-8e18-2bb9fa5155e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:24.356802 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-79f831c1-e1b0-4e92-8e18-2bb9fa5155e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:24.415789 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-07b31487-6189-4892-aed4-497f16d96a9e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:24.433226 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-79f831c1-e1b0-4e92-8e18-2bb9fa5155e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:24.441428 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-07b31487-6189-4892-aed4-497f16d96a9e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1301 microversion: 2.1 time: 0.119395 May 28 03:36:24.442030 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 47/82] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:24 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1301 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:24.449287 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-79f831c1-e1b0-4e92-8e18-2bb9fa5155e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1274 microversion: 2.1 time: 0.119906 May 28 03:36:24.449973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 36/83] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:24 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1274 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:24.451653 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9f3a4a12-8fdc-4428-a252-4f7dd8e86194 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:24.452270 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9f3a4a12-8fdc-4428-a252-4f7dd8e86194 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:24.458949 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9f3a4a12-8fdc-4428-a252-4f7dd8e86194 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:24.465057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9f3a4a12-8fdc-4428-a252-4f7dd8e86194 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:24.465057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9f3a4a12-8fdc-4428-a252-4f7dd8e86194 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:24.465057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9f3a4a12-8fdc-4428-a252-4f7dd8e86194 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:24.522204 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9f3a4a12-8fdc-4428-a252-4f7dd8e86194 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:24.529745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9f3a4a12-8fdc-4428-a252-4f7dd8e86194 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1233 microversion: 2.1 time: 0.086733 May 28 03:36:24.530317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 48/84] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:24 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1233 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:25.344044 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-06f83954-bc5a-4192-864f-fbeb30e48a77 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:25.345508 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-06f83954-bc5a-4192-864f-fbeb30e48a77 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:25.347123 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6f7fd569-4850-43d8-afd2-a381e2dbf3b0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:25.353040 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6f7fd569-4850-43d8-afd2-a381e2dbf3b0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:25.363968 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06f83954-bc5a-4192-864f-fbeb30e48a77 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:25.365602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06f83954-bc5a-4192-864f-fbeb30e48a77 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:25.366908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f7fd569-4850-43d8-afd2-a381e2dbf3b0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:25.368022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06f83954-bc5a-4192-864f-fbeb30e48a77 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:25.368781 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f7fd569-4850-43d8-afd2-a381e2dbf3b0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:25.369332 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06f83954-bc5a-4192-864f-fbeb30e48a77 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:25.372103 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f7fd569-4850-43d8-afd2-a381e2dbf3b0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:25.374715 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f7fd569-4850-43d8-afd2-a381e2dbf3b0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:25.725382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=4 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:25.789224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6f7fd569-4850-43d8-afd2-a381e2dbf3b0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:25.803638 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6f7fd569-4850-43d8-afd2-a381e2dbf3b0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:25.805830 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6f7fd569-4850-43d8-afd2-a381e2dbf3b0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:25.807162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6f7fd569-4850-43d8-afd2-a381e2dbf3b0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.474731 May 28 03:36:25.808081 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 37/85] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:25 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:25.814814 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4fee7a0d-5150-462a-8e74-48256b0a7cbc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:25.817930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4fee7a0d-5150-462a-8e74-48256b0a7cbc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:25.839030 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4fee7a0d-5150-462a-8e74-48256b0a7cbc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:25.839842 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4fee7a0d-5150-462a-8e74-48256b0a7cbc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:25.840861 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4fee7a0d-5150-462a-8e74-48256b0a7cbc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:25.841526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4fee7a0d-5150-462a-8e74-48256b0a7cbc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:25.878611 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-06f83954-bc5a-4192-864f-fbeb30e48a77 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:25.887575 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-06f83954-bc5a-4192-864f-fbeb30e48a77 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.547523 May 28 03:36:25.894902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 49/86] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:25 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:25.904983 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c1d838a3-b76c-4e5f-9efb-d9a52c9496bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:25.905723 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c1d838a3-b76c-4e5f-9efb-d9a52c9496bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:25.921342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c1d838a3-b76c-4e5f-9efb-d9a52c9496bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:25.921802 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c1d838a3-b76c-4e5f-9efb-d9a52c9496bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:25.922587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c1d838a3-b76c-4e5f-9efb-d9a52c9496bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:25.923018 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c1d838a3-b76c-4e5f-9efb-d9a52c9496bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:26.052697 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c1d838a3-b76c-4e5f-9efb-d9a52c9496bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:26.064616 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c1d838a3-b76c-4e5f-9efb-d9a52c9496bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1292 microversion: 2.1 time: 0.168230 May 28 03:36:26.066712 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 50/87] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:25 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1292 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:26.086016 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3060153c-814c-4c94-83e4-6542acbe32af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:26.089626 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3060153c-814c-4c94-83e4-6542acbe32af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:26.099640 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3060153c-814c-4c94-83e4-6542acbe32af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:26.100814 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3060153c-814c-4c94-83e4-6542acbe32af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:26.102335 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3060153c-814c-4c94-83e4-6542acbe32af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:26.102984 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3060153c-814c-4c94-83e4-6542acbe32af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:26.203636 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3060153c-814c-4c94-83e4-6542acbe32af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:26.226684 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3060153c-814c-4c94-83e4-6542acbe32af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1289 microversion: 2.1 time: 0.158746 May 28 03:36:26.236985 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4fee7a0d-5150-462a-8e74-48256b0a7cbc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:26.245491 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 51/88] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:26 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1289 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:26.246573 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4fee7a0d-5150-462a-8e74-48256b0a7cbc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1299 microversion: 2.1 time: 0.437240 May 28 03:36:26.247146 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 38/89] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:25 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1299 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:26.826651 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3bf6b0e2-712e-43bf-9e58-e952c6e6f722 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:26.827937 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3bf6b0e2-712e-43bf-9e58-e952c6e6f722 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:26.836665 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3bf6b0e2-712e-43bf-9e58-e952c6e6f722 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:26.837499 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3bf6b0e2-712e-43bf-9e58-e952c6e6f722 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:26.838475 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3bf6b0e2-712e-43bf-9e58-e952c6e6f722 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:26.839106 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3bf6b0e2-712e-43bf-9e58-e952c6e6f722 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:26.911626 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-13c3c3bb-11f9-4da6-9af8-c72271b8814d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:26.912790 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-13c3c3bb-11f9-4da6-9af8-c72271b8814d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:26.944611 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-13c3c3bb-11f9-4da6-9af8-c72271b8814d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:26.945968 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-13c3c3bb-11f9-4da6-9af8-c72271b8814d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:26.947088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-13c3c3bb-11f9-4da6-9af8-c72271b8814d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:26.947942 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-13c3c3bb-11f9-4da6-9af8-c72271b8814d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:27.275183 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3bf6b0e2-712e-43bf-9e58-e952c6e6f722 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:27.288336 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3bf6b0e2-712e-43bf-9e58-e952c6e6f722 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:27.292749 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3bf6b0e2-712e-43bf-9e58-e952c6e6f722 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:27.296694 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3bf6b0e2-712e-43bf-9e58-e952c6e6f722 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.474372 May 28 03:36:27.300000 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 52/90] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:26 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:27.311464 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3e2cca32-19b9-4203-b2f2-dae2dc1f91ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:27.312131 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3e2cca32-19b9-4203-b2f2-dae2dc1f91ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:27.327374 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3e2cca32-19b9-4203-b2f2-dae2dc1f91ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:27.328258 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3e2cca32-19b9-4203-b2f2-dae2dc1f91ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:27.329294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3e2cca32-19b9-4203-b2f2-dae2dc1f91ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:27.332810 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3e2cca32-19b9-4203-b2f2-dae2dc1f91ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:27.419507 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-13c3c3bb-11f9-4da6-9af8-c72271b8814d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:27.432551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-13c3c3bb-11f9-4da6-9af8-c72271b8814d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.528203 May 28 03:36:27.439663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 39/91] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:26 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:27.443527 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3e2cca32-19b9-4203-b2f2-dae2dc1f91ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:27.446981 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d26d7477-2889-452a-98aa-bf8dc6e33ed0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:27.447804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d26d7477-2889-452a-98aa-bf8dc6e33ed0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:27.453998 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3e2cca32-19b9-4203-b2f2-dae2dc1f91ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1280 microversion: 2.1 time: 0.152370 May 28 03:36:27.457307 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 53/92] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:27 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1280 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:27.460737 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d26d7477-2889-452a-98aa-bf8dc6e33ed0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:27.461765 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d26d7477-2889-452a-98aa-bf8dc6e33ed0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:27.462808 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d26d7477-2889-452a-98aa-bf8dc6e33ed0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:27.463387 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d26d7477-2889-452a-98aa-bf8dc6e33ed0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:27.469723 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8a5cd81f-80f7-474f-8363-37336a334838 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:27.471833 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8a5cd81f-80f7-474f-8363-37336a334838 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:27.487701 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8a5cd81f-80f7-474f-8363-37336a334838 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:27.489885 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8a5cd81f-80f7-474f-8363-37336a334838 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:27.491021 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8a5cd81f-80f7-474f-8363-37336a334838 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:27.491619 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8a5cd81f-80f7-474f-8363-37336a334838 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:27.581694 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d26d7477-2889-452a-98aa-bf8dc6e33ed0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:27.589725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d26d7477-2889-452a-98aa-bf8dc6e33ed0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1295 microversion: 2.1 time: 0.154011 May 28 03:36:27.594708 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 40/93] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:27 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1295 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:27.702823 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-9be2c2f6-0bd1-4d2b-9cc3-e41ce0ce33ea req-bc875155-2960-475f-9ff7-8e6ff9197285 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35a9c645-590f-49e1-ab03-af4167f10547", "name": "network-vif-plugged", "status": "completed", "tag": "97df7472-c1e5-4bbd-80b5-6efb1965aeb9"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:27.719939 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-9be2c2f6-0bd1-4d2b-9cc3-e41ce0ce33ea req-bc875155-2960-475f-9ff7-8e6ff9197285 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:27.720609 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-9be2c2f6-0bd1-4d2b-9cc3-e41ce0ce33ea req-bc875155-2960-475f-9ff7-8e6ff9197285 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:27.734556 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-9be2c2f6-0bd1-4d2b-9cc3-e41ce0ce33ea req-bc875155-2960-475f-9ff7-8e6ff9197285 service nova] Creating event network-vif-plugged:97df7472-c1e5-4bbd-80b5-6efb1965aeb9 for instance 35a9c645-590f-49e1-ab03-af4167f10547 on hv-cinder-84336 May 28 03:36:27.743523 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-9be2c2f6-0bd1-4d2b-9cc3-e41ce0ce33ea req-bc875155-2960-475f-9ff7-8e6ff9197285 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.149419 May 28 03:36:27.743523 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 41/94] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:27.782006 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8a5cd81f-80f7-474f-8363-37336a334838 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:27.786268 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:27.796949 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8a5cd81f-80f7-474f-8363-37336a334838 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.335578 May 28 03:36:27.800975 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 54/95] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:27 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:28.320581 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-87de56b5-0c85-46f3-88e8-eaacd5e10b38 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:28.321919 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-87de56b5-0c85-46f3-88e8-eaacd5e10b38 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:28.341875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87de56b5-0c85-46f3-88e8-eaacd5e10b38 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:28.343224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87de56b5-0c85-46f3-88e8-eaacd5e10b38 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:28.349010 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87de56b5-0c85-46f3-88e8-eaacd5e10b38 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:28.350619 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87de56b5-0c85-46f3-88e8-eaacd5e10b38 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:28.371848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-f0a2ff10-0700-418b-8241-69a2cb4df39f req-e9e87f34-5af5-48d6-8940-9710c3c5f0d3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "161bde4a-7b5a-4c34-9cad-add4855d9d1f", "tag": "a44b7ed2-6364-44d9-9fbc-d45c00c0413e"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:28.392008 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-f0a2ff10-0700-418b-8241-69a2cb4df39f req-e9e87f34-5af5-48d6-8940-9710c3c5f0d3 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:28.392458 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-f0a2ff10-0700-418b-8241-69a2cb4df39f req-e9e87f34-5af5-48d6-8940-9710c3c5f0d3 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:28.439031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-f0a2ff10-0700-418b-8241-69a2cb4df39f req-e9e87f34-5af5-48d6-8940-9710c3c5f0d3 service nova] Creating event network-changed:a44b7ed2-6364-44d9-9fbc-d45c00c0413e for instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f on hv-cinder-84336 May 28 03:36:28.461814 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-f0a2ff10-0700-418b-8241-69a2cb4df39f req-e9e87f34-5af5-48d6-8940-9710c3c5f0d3 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.094081 May 28 03:36:28.476849 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 55/96] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:36:28.484499 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-af2391d7-a1d3-4629-8e51-d62929fdbdc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:28.505257 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-af2391d7-a1d3-4629-8e51-d62929fdbdc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:28.529927 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af2391d7-a1d3-4629-8e51-d62929fdbdc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:28.530452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af2391d7-a1d3-4629-8e51-d62929fdbdc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:28.531179 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af2391d7-a1d3-4629-8e51-d62929fdbdc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:28.531618 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af2391d7-a1d3-4629-8e51-d62929fdbdc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:28.907311 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-87de56b5-0c85-46f3-88e8-eaacd5e10b38 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:28.917576 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-87de56b5-0c85-46f3-88e8-eaacd5e10b38 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:28.921775 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-87de56b5-0c85-46f3-88e8-eaacd5e10b38 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:28.923403 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-87de56b5-0c85-46f3-88e8-eaacd5e10b38 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.608595 May 28 03:36:28.924167 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 42/97] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:28 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:28.939750 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2446a84d-e6fb-4771-88ca-f1ba79cc1cc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:28.941769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2446a84d-e6fb-4771-88ca-f1ba79cc1cc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:28.949768 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2446a84d-e6fb-4771-88ca-f1ba79cc1cc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:28.950573 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2446a84d-e6fb-4771-88ca-f1ba79cc1cc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:28.951744 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2446a84d-e6fb-4771-88ca-f1ba79cc1cc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:28.952396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2446a84d-e6fb-4771-88ca-f1ba79cc1cc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:28.972954 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-af2391d7-a1d3-4629-8e51-d62929fdbdc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:28.988342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-af2391d7-a1d3-4629-8e51-d62929fdbdc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.509643 May 28 03:36:28.992073 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 56/98] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:28 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:28.996972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-cc63b4a2-2e40-4e51-8fcd-6f56df452ab9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:29.004813 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-cc63b4a2-2e40-4e51-8fcd-6f56df452ab9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:29.013029 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cc63b4a2-2e40-4e51-8fcd-6f56df452ab9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:29.013848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cc63b4a2-2e40-4e51-8fcd-6f56df452ab9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:29.014848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cc63b4a2-2e40-4e51-8fcd-6f56df452ab9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:29.015464 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cc63b4a2-2e40-4e51-8fcd-6f56df452ab9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:29.460931 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-cc63b4a2-2e40-4e51-8fcd-6f56df452ab9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:29.469129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-cc63b4a2-2e40-4e51-8fcd-6f56df452ab9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.475806 May 28 03:36:29.469721 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 57/99] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:28 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:29.474936 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-256c1235-f4c9-49b4-ba3e-c3fdfe64e22e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:29.475711 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-256c1235-f4c9-49b4-ba3e-c3fdfe64e22e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:29.485426 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-256c1235-f4c9-49b4-ba3e-c3fdfe64e22e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:29.487539 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-256c1235-f4c9-49b4-ba3e-c3fdfe64e22e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:29.488835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-256c1235-f4c9-49b4-ba3e-c3fdfe64e22e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:29.489316 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-256c1235-f4c9-49b4-ba3e-c3fdfe64e22e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:29.525662 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2446a84d-e6fb-4771-88ca-f1ba79cc1cc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:29.543621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2446a84d-e6fb-4771-88ca-f1ba79cc1cc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.617654 May 28 03:36:29.557228 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 43/100] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:28 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:29.699086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-256c1235-f4c9-49b4-ba3e-c3fdfe64e22e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:29.708102 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-256c1235-f4c9-49b4-ba3e-c3fdfe64e22e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.237072 May 28 03:36:29.710480 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 58/101] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:29 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:29.760028 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-bdefb953-1294-4f20-bfb8-6eb7b8fe0dc5 req-0d19eb80-6bc8-44f8-bd6c-1b241f7d28c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "362687d7-ef88-4e49-b7a5-d732c28e1b0b", "name": "network-vif-plugged", "status": "completed", "tag": "cda792d3-ee73-4947-a077-2caf88494a15"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:29.774484 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-bdefb953-1294-4f20-bfb8-6eb7b8fe0dc5 req-0d19eb80-6bc8-44f8-bd6c-1b241f7d28c3 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:29.780311 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-bdefb953-1294-4f20-bfb8-6eb7b8fe0dc5 req-0d19eb80-6bc8-44f8-bd6c-1b241f7d28c3 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:29.802804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-bdefb953-1294-4f20-bfb8-6eb7b8fe0dc5 req-0d19eb80-6bc8-44f8-bd6c-1b241f7d28c3 service nova] Creating event network-vif-plugged:cda792d3-ee73-4947-a077-2caf88494a15 for instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b on hv-cinder-84336 May 28 03:36:29.811473 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-bdefb953-1294-4f20-bfb8-6eb7b8fe0dc5 req-0d19eb80-6bc8-44f8-bd6c-1b241f7d28c3 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055131 May 28 03:36:29.813032 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 44/102] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:36:29.942566 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dd77a2fc-0409-49bf-b82c-be0616ab6ac8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:29.942958 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dd77a2fc-0409-49bf-b82c-be0616ab6ac8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:29.944961 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=5 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:29.950799 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd77a2fc-0409-49bf-b82c-be0616ab6ac8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:29.951255 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd77a2fc-0409-49bf-b82c-be0616ab6ac8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:29.952031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd77a2fc-0409-49bf-b82c-be0616ab6ac8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:29.952468 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd77a2fc-0409-49bf-b82c-be0616ab6ac8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:30.009476 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4c39c3ae-8114-4d28-8a78-0e72ca14571d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:30.010351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4c39c3ae-8114-4d28-8a78-0e72ca14571d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:30.017338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c39c3ae-8114-4d28-8a78-0e72ca14571d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:30.018229 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c39c3ae-8114-4d28-8a78-0e72ca14571d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:30.019319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c39c3ae-8114-4d28-8a78-0e72ca14571d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:30.019920 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c39c3ae-8114-4d28-8a78-0e72ca14571d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:30.024325 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=100 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:30.095096 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dd77a2fc-0409-49bf-b82c-be0616ab6ac8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:30.105458 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dd77a2fc-0409-49bf-b82c-be0616ab6ac8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:30.106693 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dd77a2fc-0409-49bf-b82c-be0616ab6ac8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:30.107878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-dd77a2fc-0409-49bf-b82c-be0616ab6ac8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.170379 May 28 03:36:30.113817 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 59/103] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:29 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:30.368448 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4c39c3ae-8114-4d28-8a78-0e72ca14571d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:30.382068 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4c39c3ae-8114-4d28-8a78-0e72ca14571d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.377233 May 28 03:36:30.383810 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 45/104] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:30 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:36:30.486148 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f0e555ac-4274-4799-a69c-615cdf7f206f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:30.487201 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f0e555ac-4274-4799-a69c-615cdf7f206f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:30.493346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f0e555ac-4274-4799-a69c-615cdf7f206f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:30.493794 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f0e555ac-4274-4799-a69c-615cdf7f206f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:30.494962 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f0e555ac-4274-4799-a69c-615cdf7f206f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:30.495404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f0e555ac-4274-4799-a69c-615cdf7f206f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:30.562835 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-da62c66d-2096-446f-8994-0b614628e5d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:30.563443 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-da62c66d-2096-446f-8994-0b614628e5d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:30.570207 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-da62c66d-2096-446f-8994-0b614628e5d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:30.570669 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-da62c66d-2096-446f-8994-0b614628e5d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:30.571444 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-da62c66d-2096-446f-8994-0b614628e5d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:30.571913 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-da62c66d-2096-446f-8994-0b614628e5d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:30.659564 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f0e555ac-4274-4799-a69c-615cdf7f206f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:30.671787 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f0e555ac-4274-4799-a69c-615cdf7f206f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.189119 May 28 03:36:30.672232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 60/105] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:30 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:36:30.713428 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-37c0e7d8-f55e-4cc9-9bb8-16af96b2cc7e req-dd1254c4-c3af-42ba-abcf-c0693e8dc0c3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "da053642-eac6-4484-9557-f0b878712cf2", "tag": "63e23069-fb2c-4804-8da1-bf97b736bc08"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:30.726077 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-37c0e7d8-f55e-4cc9-9bb8-16af96b2cc7e req-dd1254c4-c3af-42ba-abcf-c0693e8dc0c3 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:30.726786 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-37c0e7d8-f55e-4cc9-9bb8-16af96b2cc7e req-dd1254c4-c3af-42ba-abcf-c0693e8dc0c3 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:30.742411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-37c0e7d8-f55e-4cc9-9bb8-16af96b2cc7e req-dd1254c4-c3af-42ba-abcf-c0693e8dc0c3 service nova] Creating event network-changed:63e23069-fb2c-4804-8da1-bf97b736bc08 for instance da053642-eac6-4484-9557-f0b878712cf2 on hv-cinder-84336 May 28 03:36:30.753213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-37c0e7d8-f55e-4cc9-9bb8-16af96b2cc7e req-dd1254c4-c3af-42ba-abcf-c0693e8dc0c3 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039789 May 28 03:36:30.753213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 61/106] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:36:30.757836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-16e1e835-e55c-4841-81da-2dc243118b99 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:30.758542 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-16e1e835-e55c-4841-81da-2dc243118b99 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:30.770736 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-16e1e835-e55c-4841-81da-2dc243118b99 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:30.771426 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-16e1e835-e55c-4841-81da-2dc243118b99 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:30.772223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-16e1e835-e55c-4841-81da-2dc243118b99 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:30.773283 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-16e1e835-e55c-4841-81da-2dc243118b99 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:30.831590 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-da62c66d-2096-446f-8994-0b614628e5d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:30.850491 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-da62c66d-2096-446f-8994-0b614628e5d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.291609 May 28 03:36:30.871544 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 46/107] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:30 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:36:30.955176 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-16e1e835-e55c-4841-81da-2dc243118b99 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:30.964012 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-16e1e835-e55c-4841-81da-2dc243118b99 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.212280 May 28 03:36:30.965167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 62/108] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:30 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:31.128465 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1ac0dd7f-9e7a-48ce-926a-b30c21edd155 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:31.129200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1ac0dd7f-9e7a-48ce-926a-b30c21edd155 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:31.135273 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ac0dd7f-9e7a-48ce-926a-b30c21edd155 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:31.135723 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ac0dd7f-9e7a-48ce-926a-b30c21edd155 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:31.136483 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ac0dd7f-9e7a-48ce-926a-b30c21edd155 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:31.136920 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ac0dd7f-9e7a-48ce-926a-b30c21edd155 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:31.266786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1ac0dd7f-9e7a-48ce-926a-b30c21edd155 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:31.274918 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1ac0dd7f-9e7a-48ce-926a-b30c21edd155 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:31.276398 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1ac0dd7f-9e7a-48ce-926a-b30c21edd155 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:31.277517 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1ac0dd7f-9e7a-48ce-926a-b30c21edd155 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.153217 May 28 03:36:31.278182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 47/109] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:31 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:31.398723 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a61cd388-2bf6-4a9b-a2d5-429d69ea648d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:31.399318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a61cd388-2bf6-4a9b-a2d5-429d69ea648d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:31.405325 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a61cd388-2bf6-4a9b-a2d5-429d69ea648d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:31.405766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a61cd388-2bf6-4a9b-a2d5-429d69ea648d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:31.406517 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a61cd388-2bf6-4a9b-a2d5-429d69ea648d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:31.406942 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a61cd388-2bf6-4a9b-a2d5-429d69ea648d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:31.538296 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a61cd388-2bf6-4a9b-a2d5-429d69ea648d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:31.546104 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a61cd388-2bf6-4a9b-a2d5-429d69ea648d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.151224 May 28 03:36:31.546681 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 63/110] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:31 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:31.689034 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-60516205-b775-4cee-b638-2ab125547549 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:31.690386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-60516205-b775-4cee-b638-2ab125547549 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:31.691782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:31.692415 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:31.697280 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-60516205-b775-4cee-b638-2ab125547549 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:31.697726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-60516205-b775-4cee-b638-2ab125547549 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:31.698546 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-60516205-b775-4cee-b638-2ab125547549 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:31.698976 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-60516205-b775-4cee-b638-2ab125547549 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:31.833078 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-60516205-b775-4cee-b638-2ab125547549 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:31.840459 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-60516205-b775-4cee-b638-2ab125547549 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.155434 May 28 03:36:31.841021 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 48/111] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:31 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:31.888205 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3957068c-b93c-41aa-8a01-c5755195028a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:31.888809 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3957068c-b93c-41aa-8a01-c5755195028a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:31.895386 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3957068c-b93c-41aa-8a01-c5755195028a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:31.895878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3957068c-b93c-41aa-8a01-c5755195028a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:31.896633 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3957068c-b93c-41aa-8a01-c5755195028a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:31.897067 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3957068c-b93c-41aa-8a01-c5755195028a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:31.981214 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-213328b5-4f1f-4f2f-85cc-abdbb0929ca3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:31.981824 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-213328b5-4f1f-4f2f-85cc-abdbb0929ca3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:31.988170 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-213328b5-4f1f-4f2f-85cc-abdbb0929ca3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:31.988550 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-213328b5-4f1f-4f2f-85cc-abdbb0929ca3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:31.989414 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-213328b5-4f1f-4f2f-85cc-abdbb0929ca3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:31.989852 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-213328b5-4f1f-4f2f-85cc-abdbb0929ca3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:32.031623 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3957068c-b93c-41aa-8a01-c5755195028a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:32.041435 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3957068c-b93c-41aa-8a01-c5755195028a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.156775 May 28 03:36:32.042189 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 64/112] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:31 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:32.126001 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-213328b5-4f1f-4f2f-85cc-abdbb0929ca3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:32.134086 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-213328b5-4f1f-4f2f-85cc-abdbb0929ca3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.156856 May 28 03:36:32.135096 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 49/113] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:31 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:36:32.296829 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-afed061a-e511-4e98-aafb-c6e039e17de7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:32.297473 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-afed061a-e511-4e98-aafb-c6e039e17de7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:32.304264 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-afed061a-e511-4e98-aafb-c6e039e17de7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:32.310195 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-afed061a-e511-4e98-aafb-c6e039e17de7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:32.310195 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-afed061a-e511-4e98-aafb-c6e039e17de7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:32.310195 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-afed061a-e511-4e98-aafb-c6e039e17de7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:32.470265 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-afed061a-e511-4e98-aafb-c6e039e17de7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:32.480030 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-afed061a-e511-4e98-aafb-c6e039e17de7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:32.481575 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-afed061a-e511-4e98-aafb-c6e039e17de7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:32.484881 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-afed061a-e511-4e98-aafb-c6e039e17de7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.190797 May 28 03:36:32.484881 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 65/114] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:32 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:36:32.562650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a5a7bebf-b401-4828-b832-94caf9257d4d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:32.563204 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a5a7bebf-b401-4828-b832-94caf9257d4d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:32.569137 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a5a7bebf-b401-4828-b832-94caf9257d4d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:32.569594 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a5a7bebf-b401-4828-b832-94caf9257d4d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:32.570344 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a5a7bebf-b401-4828-b832-94caf9257d4d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:32.570774 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a5a7bebf-b401-4828-b832-94caf9257d4d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:32.792354 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-7dc48544-6aec-4784-a38d-de1163e5660e req-bed627a6-375a-454c-80d5-7ea54c1c0726 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "247082dc-5284-462d-92c8-045154d864e4", "tag": "6aaa9bcc-6545-4d6b-95af-ee421ef5771b"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:32.801289 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-7dc48544-6aec-4784-a38d-de1163e5660e req-bed627a6-375a-454c-80d5-7ea54c1c0726 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:32.804901 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-7dc48544-6aec-4784-a38d-de1163e5660e req-bed627a6-375a-454c-80d5-7ea54c1c0726 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:32.810630 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a5a7bebf-b401-4828-b832-94caf9257d4d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:32.814555 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-7dc48544-6aec-4784-a38d-de1163e5660e req-bed627a6-375a-454c-80d5-7ea54c1c0726 service nova] Creating event network-changed:6aaa9bcc-6545-4d6b-95af-ee421ef5771b for instance 247082dc-5284-462d-92c8-045154d864e4 on hv-cinder-84336 May 28 03:36:32.819823 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a5a7bebf-b401-4828-b832-94caf9257d4d tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.260954 May 28 03:36:32.819995 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-7dc48544-6aec-4784-a38d-de1163e5660e req-bed627a6-375a-454c-80d5-7ea54c1c0726 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031199 May 28 03:36:32.820450 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 66/115] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:36:32.821297 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 50/116] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:32 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:36:32.858282 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-24558030-156b-43a7-9b3b-6eebe7a4ec4e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:32.858974 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-24558030-156b-43a7-9b3b-6eebe7a4ec4e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:32.865507 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24558030-156b-43a7-9b3b-6eebe7a4ec4e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:32.865972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24558030-156b-43a7-9b3b-6eebe7a4ec4e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:32.866731 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24558030-156b-43a7-9b3b-6eebe7a4ec4e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:32.867155 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24558030-156b-43a7-9b3b-6eebe7a4ec4e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:33.014284 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-24558030-156b-43a7-9b3b-6eebe7a4ec4e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:33.022714 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-24558030-156b-43a7-9b3b-6eebe7a4ec4e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.168548 May 28 03:36:33.023292 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 67/117] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:32 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:33.059461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-21bf2b60-97f5-43cb-a55a-cbd2aa6b5125 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:33.060071 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-21bf2b60-97f5-43cb-a55a-cbd2aa6b5125 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:33.066752 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-21bf2b60-97f5-43cb-a55a-cbd2aa6b5125 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:33.067126 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-21bf2b60-97f5-43cb-a55a-cbd2aa6b5125 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:33.067890 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-21bf2b60-97f5-43cb-a55a-cbd2aa6b5125 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:33.068496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-21bf2b60-97f5-43cb-a55a-cbd2aa6b5125 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:33.155365 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-90cf71da-fb45-4d9d-9cc3-d9151708f704 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:33.156346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-90cf71da-fb45-4d9d-9cc3-d9151708f704 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:33.163575 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90cf71da-fb45-4d9d-9cc3-d9151708f704 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:33.164090 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90cf71da-fb45-4d9d-9cc3-d9151708f704 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:33.164920 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90cf71da-fb45-4d9d-9cc3-d9151708f704 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:33.165448 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90cf71da-fb45-4d9d-9cc3-d9151708f704 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:33.337653 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-90cf71da-fb45-4d9d-9cc3-d9151708f704 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:33.346050 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-90cf71da-fb45-4d9d-9cc3-d9151708f704 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.197463 May 28 03:36:33.353855 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 68/118] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:33 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:33.356102 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-21bf2b60-97f5-43cb-a55a-cbd2aa6b5125 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:33.369741 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-21bf2b60-97f5-43cb-a55a-cbd2aa6b5125 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.308617 May 28 03:36:33.369741 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 51/119] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:33 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:33.500843 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a056c323-ce2d-4747-b675-a95a0e452729 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:33.504095 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a056c323-ce2d-4747-b675-a95a0e452729 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:33.511743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a056c323-ce2d-4747-b675-a95a0e452729 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:33.511743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a056c323-ce2d-4747-b675-a95a0e452729 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:33.511743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a056c323-ce2d-4747-b675-a95a0e452729 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:33.511743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a056c323-ce2d-4747-b675-a95a0e452729 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:33.653522 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a056c323-ce2d-4747-b675-a95a0e452729 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:33.665066 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a056c323-ce2d-4747-b675-a95a0e452729 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:33.666723 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a056c323-ce2d-4747-b675-a95a0e452729 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:33.667919 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a056c323-ce2d-4747-b675-a95a0e452729 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1408 microversion: 2.20 time: 0.171071 May 28 03:36:33.669221 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 69/120] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:33 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1408 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:33.838468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-43ba124d-652e-4327-91df-208e51276baa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:33.839184 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-43ba124d-652e-4327-91df-208e51276baa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:33.851944 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-43ba124d-652e-4327-91df-208e51276baa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:33.852395 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-43ba124d-652e-4327-91df-208e51276baa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:33.853241 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-43ba124d-652e-4327-91df-208e51276baa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:33.853682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-43ba124d-652e-4327-91df-208e51276baa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:33.989405 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-43ba124d-652e-4327-91df-208e51276baa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:33.997049 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-43ba124d-652e-4327-91df-208e51276baa tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1314 microversion: 2.1 time: 0.162621 May 28 03:36:33.997644 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 52/121] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:33 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:34.039724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f244adcb-0547-4b8e-a8ad-a62a55845e9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:34.040330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f244adcb-0547-4b8e-a8ad-a62a55845e9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:34.046348 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f244adcb-0547-4b8e-a8ad-a62a55845e9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:34.046905 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f244adcb-0547-4b8e-a8ad-a62a55845e9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:34.047676 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f244adcb-0547-4b8e-a8ad-a62a55845e9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:34.049648 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f244adcb-0547-4b8e-a8ad-a62a55845e9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:34.202501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f244adcb-0547-4b8e-a8ad-a62a55845e9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:34.211599 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f244adcb-0547-4b8e-a8ad-a62a55845e9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.175611 May 28 03:36:34.212753 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 70/122] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:34 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:34.368002 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4c51fec9-46eb-4c0a-96f4-5304e462259b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:34.368650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4c51fec9-46eb-4c0a-96f4-5304e462259b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:34.381191 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c51fec9-46eb-4c0a-96f4-5304e462259b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:34.381191 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c51fec9-46eb-4c0a-96f4-5304e462259b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:34.381191 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c51fec9-46eb-4c0a-96f4-5304e462259b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:34.381191 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c51fec9-46eb-4c0a-96f4-5304e462259b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:34.384239 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e07774ed-dbcb-4549-a586-0c8133186faf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:34.384847 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e07774ed-dbcb-4549-a586-0c8133186faf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:34.391060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e07774ed-dbcb-4549-a586-0c8133186faf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:34.391511 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e07774ed-dbcb-4549-a586-0c8133186faf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:34.392289 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e07774ed-dbcb-4549-a586-0c8133186faf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:34.392889 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e07774ed-dbcb-4549-a586-0c8133186faf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:34.555780 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4c51fec9-46eb-4c0a-96f4-5304e462259b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:34.563550 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4c51fec9-46eb-4c0a-96f4-5304e462259b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.204475 May 28 03:36:34.564193 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 53/123] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:34 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:34.568498 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-58fa45e5-3d7f-45db-85b1-955ec8c1ab3b req-1677d8d7-59cc-46a6-9604-ae7a12282c1b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b4a0f56-eb59-4f13-a90a-442b9a2e0af8", "name": "network-vif-plugged", "status": "completed", "tag": "2038175b-b88a-437c-9114-63208c464911"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:34.573145 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:34.577405 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-58fa45e5-3d7f-45db-85b1-955ec8c1ab3b req-1677d8d7-59cc-46a6-9604-ae7a12282c1b service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:34.577850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-58fa45e5-3d7f-45db-85b1-955ec8c1ab3b req-1677d8d7-59cc-46a6-9604-ae7a12282c1b service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:34.581984 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e07774ed-dbcb-4549-a586-0c8133186faf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:34.590276 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-58fa45e5-3d7f-45db-85b1-955ec8c1ab3b req-1677d8d7-59cc-46a6-9604-ae7a12282c1b service nova] Creating event network-vif-plugged:2038175b-b88a-437c-9114-63208c464911 for instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 on hv-cinder-84336 May 28 03:36:34.592294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e07774ed-dbcb-4549-a586-0c8133186faf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.212241 May 28 03:36:34.593129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 71/124] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:34 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:34.595740 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-58fa45e5-3d7f-45db-85b1-955ec8c1ab3b req-1677d8d7-59cc-46a6-9604-ae7a12282c1b service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030435 May 28 03:36:34.596426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 54/125] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:34.686376 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d81a36ea-6be3-44ee-bb9f-6dcd6cc53c07 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:34.687002 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d81a36ea-6be3-44ee-bb9f-6dcd6cc53c07 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:34.695793 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d81a36ea-6be3-44ee-bb9f-6dcd6cc53c07 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:34.696242 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d81a36ea-6be3-44ee-bb9f-6dcd6cc53c07 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:34.697086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d81a36ea-6be3-44ee-bb9f-6dcd6cc53c07 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:34.697553 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d81a36ea-6be3-44ee-bb9f-6dcd6cc53c07 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:34.830070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d81a36ea-6be3-44ee-bb9f-6dcd6cc53c07 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:34.837929 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d81a36ea-6be3-44ee-bb9f-6dcd6cc53c07 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:34.839411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d81a36ea-6be3-44ee-bb9f-6dcd6cc53c07 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:34.840551 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d81a36ea-6be3-44ee-bb9f-6dcd6cc53c07 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1600 microversion: 2.20 time: 0.158257 May 28 03:36:34.841125 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 72/126] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:36:34 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1600 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:36:34.863050 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ee5f6b25-5c37-4313-b4dd-2d09fa62d22d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:34.863626 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ee5f6b25-5c37-4313-b4dd-2d09fa62d22d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:34.870360 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ee5f6b25-5c37-4313-b4dd-2d09fa62d22d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:34.870799 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ee5f6b25-5c37-4313-b4dd-2d09fa62d22d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:34.871700 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ee5f6b25-5c37-4313-b4dd-2d09fa62d22d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:34.872239 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ee5f6b25-5c37-4313-b4dd-2d09fa62d22d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:34.989861 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ee5f6b25-5c37-4313-b4dd-2d09fa62d22d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.132089 May 28 03:36:34.990392 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 55/127] 10.222.0.41 () {60 vars in 1315 bytes} [Sat May 28 03:36:34 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-interface => generated 284 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:36:35.015497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3a5b9abf-9d6c-4900-a879-567d9cdf3cbd tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:35.016222 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3a5b9abf-9d6c-4900-a879-567d9cdf3cbd tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:35.025310 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3a5b9abf-9d6c-4900-a879-567d9cdf3cbd tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:35.025854 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3a5b9abf-9d6c-4900-a879-567d9cdf3cbd tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:35.026627 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3a5b9abf-9d6c-4900-a879-567d9cdf3cbd tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:35.027091 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3a5b9abf-9d6c-4900-a879-567d9cdf3cbd tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:35.053386 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:35.214840 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3a5b9abf-9d6c-4900-a879-567d9cdf3cbd tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:35.224566 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3a5b9abf-9d6c-4900-a879-567d9cdf3cbd tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1500 microversion: 2.1 time: 0.212903 May 28 03:36:35.227235 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 73/128] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:35 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1500 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:35.231159 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-55453986-1e3e-4e7b-801a-67137307589d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:35.232090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-55453986-1e3e-4e7b-801a-67137307589d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:35.240102 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-55453986-1e3e-4e7b-801a-67137307589d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:35.240633 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-55453986-1e3e-4e7b-801a-67137307589d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:35.241586 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-55453986-1e3e-4e7b-801a-67137307589d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:35.242079 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-55453986-1e3e-4e7b-801a-67137307589d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:35.584657 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-200d91a5-2dc1-43fd-967e-92f91f339833 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:35.584657 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-200d91a5-2dc1-43fd-967e-92f91f339833 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:35.586657 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:35.591706 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-200d91a5-2dc1-43fd-967e-92f91f339833 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:35.592239 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-200d91a5-2dc1-43fd-967e-92f91f339833 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:35.593079 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-200d91a5-2dc1-43fd-967e-92f91f339833 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:35.593603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-200d91a5-2dc1-43fd-967e-92f91f339833 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:35.725770 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-55453986-1e3e-4e7b-801a-67137307589d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:35.741436 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-55453986-1e3e-4e7b-801a-67137307589d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.514683 May 28 03:36:35.749538 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 56/129] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:35 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:35.756639 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b61a982a-1766-496d-8997-2e14ddee6cc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:35.760415 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-b61a982a-1766-496d-8997-2e14ddee6cc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:35.769909 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b61a982a-1766-496d-8997-2e14ddee6cc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:35.770616 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b61a982a-1766-496d-8997-2e14ddee6cc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:35.771600 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b61a982a-1766-496d-8997-2e14ddee6cc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:35.772227 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b61a982a-1766-496d-8997-2e14ddee6cc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:35.805356 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-200d91a5-2dc1-43fd-967e-92f91f339833 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:35.818210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-200d91a5-2dc1-43fd-967e-92f91f339833 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.242052 May 28 03:36:35.819513 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 74/130] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:35 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:36.140773 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-b61a982a-1766-496d-8997-2e14ddee6cc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:36.149819 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-b61a982a-1766-496d-8997-2e14ddee6cc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.398719 May 28 03:36:36.162584 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 57/131] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:35 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:36.765275 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ee2a9f64-a598-43c3-9bbd-7a402e9c59b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:36.765997 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ee2a9f64-a598-43c3-9bbd-7a402e9c59b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:36.773226 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ee2a9f64-a598-43c3-9bbd-7a402e9c59b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:36.773688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ee2a9f64-a598-43c3-9bbd-7a402e9c59b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:36.774452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ee2a9f64-a598-43c3-9bbd-7a402e9c59b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:36.774885 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ee2a9f64-a598-43c3-9bbd-7a402e9c59b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:36.836073 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-79e7be96-0ee3-4ce4-8ddb-38f2e2cd4348 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:36.836850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-79e7be96-0ee3-4ce4-8ddb-38f2e2cd4348 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:36.843994 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-79e7be96-0ee3-4ce4-8ddb-38f2e2cd4348 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:36.844535 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-79e7be96-0ee3-4ce4-8ddb-38f2e2cd4348 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:36.845504 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-79e7be96-0ee3-4ce4-8ddb-38f2e2cd4348 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:36.845980 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-79e7be96-0ee3-4ce4-8ddb-38f2e2cd4348 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:37.064018 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ee2a9f64-a598-43c3-9bbd-7a402e9c59b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:37.074319 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ee2a9f64-a598-43c3-9bbd-7a402e9c59b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.312825 May 28 03:36:37.085355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 75/132] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:36 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:37.170111 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d4f76919-bee4-4367-8a07-1847c8c743da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:37.170723 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d4f76919-bee4-4367-8a07-1847c8c743da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:37.178323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4f76919-bee4-4367-8a07-1847c8c743da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:37.178774 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4f76919-bee4-4367-8a07-1847c8c743da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:37.179851 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4f76919-bee4-4367-8a07-1847c8c743da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:37.180202 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4f76919-bee4-4367-8a07-1847c8c743da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:37.279013 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-79e7be96-0ee3-4ce4-8ddb-38f2e2cd4348 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:37.293845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-79e7be96-0ee3-4ce4-8ddb-38f2e2cd4348 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.461689 May 28 03:36:37.297360 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 58/133] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:36 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:36:37.336354 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d4f76919-bee4-4367-8a07-1847c8c743da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:37.344662 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d4f76919-bee4-4367-8a07-1847c8c743da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.178746 May 28 03:36:37.345731 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 76/134] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:37 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:36:37.423877 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0d791a67-629a-4b64-9495-c6b2e64cb586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4acf7a39-855b-49f5-9e2d-c8c8550c6191"}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:37.423877 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0d791a67-629a-4b64-9495-c6b2e64cb586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:37.433574 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d791a67-629a-4b64-9495-c6b2e64cb586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:37.434729 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-59c79197-9361-4d40-893c-e03cdb857930 req-e349fb0c-96ec-4860-a490-7fdbf49db65d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35a9c645-590f-49e1-ab03-af4167f10547", "tag": "97df7472-c1e5-4bbd-80b5-6efb1965aeb9"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:37.434863 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d791a67-629a-4b64-9495-c6b2e64cb586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:37.437312 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d791a67-629a-4b64-9495-c6b2e64cb586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:37.438700 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d791a67-629a-4b64-9495-c6b2e64cb586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:37.447375 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-59c79197-9361-4d40-893c-e03cdb857930 req-e349fb0c-96ec-4860-a490-7fdbf49db65d service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:37.447821 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-59c79197-9361-4d40-893c-e03cdb857930 req-e349fb0c-96ec-4860-a490-7fdbf49db65d service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:37.463034 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-59c79197-9361-4d40-893c-e03cdb857930 req-e349fb0c-96ec-4860-a490-7fdbf49db65d service nova] Creating event network-changed:97df7472-c1e5-4bbd-80b5-6efb1965aeb9 for instance 35a9c645-590f-49e1-ab03-af4167f10547 on hv-cinder-84336 May 28 03:36:37.475462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-59c79197-9361-4d40-893c-e03cdb857930 req-e349fb0c-96ec-4860-a490-7fdbf49db65d service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045419 May 28 03:36:37.478794 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 77/135] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:36:38.091908 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9b488914-7bb5-47ac-9c7e-4a061e3bcf4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:38.093323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9b488914-7bb5-47ac-9c7e-4a061e3bcf4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:38.099866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b488914-7bb5-47ac-9c7e-4a061e3bcf4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:38.100508 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b488914-7bb5-47ac-9c7e-4a061e3bcf4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:38.105515 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b488914-7bb5-47ac-9c7e-4a061e3bcf4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:38.105515 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b488914-7bb5-47ac-9c7e-4a061e3bcf4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:38.289369 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9b488914-7bb5-47ac-9c7e-4a061e3bcf4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:38.306488 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9b488914-7bb5-47ac-9c7e-4a061e3bcf4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.218697 May 28 03:36:38.307517 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 78/136] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:38 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:38.322290 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0348deca-e3e9-47a3-b475-1fecf452e203 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:38.323217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0348deca-e3e9-47a3-b475-1fecf452e203 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:38.330950 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0348deca-e3e9-47a3-b475-1fecf452e203 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:38.331408 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0348deca-e3e9-47a3-b475-1fecf452e203 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:38.332179 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0348deca-e3e9-47a3-b475-1fecf452e203 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:38.332616 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0348deca-e3e9-47a3-b475-1fecf452e203 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:38.512782 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0348deca-e3e9-47a3-b475-1fecf452e203 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:38.522179 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0348deca-e3e9-47a3-b475-1fecf452e203 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.203593 May 28 03:36:38.523006 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 79/137] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:38 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:38.537510 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ea06e75c-ae30-43c4-8d7f-147a1aa8455d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:38.538297 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ea06e75c-ae30-43c4-8d7f-147a1aa8455d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:38.545860 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ea06e75c-ae30-43c4-8d7f-147a1aa8455d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:38.546302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ea06e75c-ae30-43c4-8d7f-147a1aa8455d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:38.547127 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ea06e75c-ae30-43c4-8d7f-147a1aa8455d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:38.547588 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ea06e75c-ae30-43c4-8d7f-147a1aa8455d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:38.700888 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ea06e75c-ae30-43c4-8d7f-147a1aa8455d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:38.713489 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ea06e75c-ae30-43c4-8d7f-147a1aa8455d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.179554 May 28 03:36:38.714683 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 80/138] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:38 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:39.324056 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-93932ca9-045c-47d8-abde-46a7a425b179 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:39.324694 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-93932ca9-045c-47d8-abde-46a7a425b179 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:39.331342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-93932ca9-045c-47d8-abde-46a7a425b179 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:39.331794 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-93932ca9-045c-47d8-abde-46a7a425b179 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:39.333166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-93932ca9-045c-47d8-abde-46a7a425b179 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:39.333602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-93932ca9-045c-47d8-abde-46a7a425b179 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:39.393562 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0d791a67-629a-4b64-9495-c6b2e64cb586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "POST /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.974431 May 28 03:36:39.401477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 59/139] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:36:37 2022] POST /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b/os-volume_attachments => generated 194 bytes in 1983 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:39.488019 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-93932ca9-045c-47d8-abde-46a7a425b179 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:39.496614 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-93932ca9-045c-47d8-abde-46a7a425b179 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.176231 May 28 03:36:39.497776 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 81/140] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:39 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:39.540571 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d3b35312-1693-48db-a713-a687c5f11208 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:39.541305 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d3b35312-1693-48db-a713-a687c5f11208 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:39.548319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3b35312-1693-48db-a713-a687c5f11208 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:39.549181 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3b35312-1693-48db-a713-a687c5f11208 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:39.550168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3b35312-1693-48db-a713-a687c5f11208 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:39.550739 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3b35312-1693-48db-a713-a687c5f11208 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:39.744965 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-71c52ded-239d-4b1d-bb33-90aaab9805d6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:39.745966 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-71c52ded-239d-4b1d-bb33-90aaab9805d6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:39.756953 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-71c52ded-239d-4b1d-bb33-90aaab9805d6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:39.757432 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-71c52ded-239d-4b1d-bb33-90aaab9805d6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:39.758203 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-71c52ded-239d-4b1d-bb33-90aaab9805d6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:39.758857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-71c52ded-239d-4b1d-bb33-90aaab9805d6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:39.788329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d3b35312-1693-48db-a713-a687c5f11208 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:39.795854 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d3b35312-1693-48db-a713-a687c5f11208 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.259717 May 28 03:36:39.805728 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 60/141] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:39 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:40.030063 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-71c52ded-239d-4b1d-bb33-90aaab9805d6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:40.042556 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-71c52ded-239d-4b1d-bb33-90aaab9805d6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.301301 May 28 03:36:40.052593 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 82/142] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:39 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:40.513926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-42fa15a6-828f-4b3e-b346-5c72d1c3e7fd tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:40.514461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-42fa15a6-828f-4b3e-b346-5c72d1c3e7fd tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:40.520481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-42fa15a6-828f-4b3e-b346-5c72d1c3e7fd tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:40.520923 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-42fa15a6-828f-4b3e-b346-5c72d1c3e7fd tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:40.521702 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-42fa15a6-828f-4b3e-b346-5c72d1c3e7fd tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:40.522279 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-42fa15a6-828f-4b3e-b346-5c72d1c3e7fd tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:40.537324 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:40.664869 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-42fa15a6-828f-4b3e-b346-5c72d1c3e7fd tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:40.683576 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-42fa15a6-828f-4b3e-b346-5c72d1c3e7fd tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.170997 May 28 03:36:40.684174 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 61/143] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:40 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:40.812383 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-adfb6ecf-89f2-45de-af6e-0b553ce64e28 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:40.813129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-adfb6ecf-89f2-45de-af6e-0b553ce64e28 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:40.820271 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-adfb6ecf-89f2-45de-af6e-0b553ce64e28 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:40.820718 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-adfb6ecf-89f2-45de-af6e-0b553ce64e28 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:40.821618 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-adfb6ecf-89f2-45de-af6e-0b553ce64e28 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:40.822063 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-adfb6ecf-89f2-45de-af6e-0b553ce64e28 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:40.959740 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-adfb6ecf-89f2-45de-af6e-0b553ce64e28 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:40.970360 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-adfb6ecf-89f2-45de-af6e-0b553ce64e28 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.162250 May 28 03:36:40.971657 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 83/144] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:40 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:41.059578 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a98679b3-53ba-4acf-ad02-282da7e5b31b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:41.060195 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a98679b3-53ba-4acf-ad02-282da7e5b31b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:41.068917 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a98679b3-53ba-4acf-ad02-282da7e5b31b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:41.069405 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a98679b3-53ba-4acf-ad02-282da7e5b31b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:41.070241 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a98679b3-53ba-4acf-ad02-282da7e5b31b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:41.070739 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a98679b3-53ba-4acf-ad02-282da7e5b31b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:41.206133 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a98679b3-53ba-4acf-ad02-282da7e5b31b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:41.213517 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a98679b3-53ba-4acf-ad02-282da7e5b31b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.157712 May 28 03:36:41.214292 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 62/145] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:41 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:41.474273 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-5f43dd71-7518-4e0f-8b52-91303427fba1 req-2aaf1d28-55c3-49e9-b981-91bc8f044685 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "161bde4a-7b5a-4c34-9cad-add4855d9d1f", "name": "network-vif-plugged", "status": "completed", "tag": "a44b7ed2-6364-44d9-9fbc-d45c00c0413e"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:41.482398 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-5f43dd71-7518-4e0f-8b52-91303427fba1 req-2aaf1d28-55c3-49e9-b981-91bc8f044685 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:41.482840 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-5f43dd71-7518-4e0f-8b52-91303427fba1 req-2aaf1d28-55c3-49e9-b981-91bc8f044685 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:41.494649 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-5f43dd71-7518-4e0f-8b52-91303427fba1 req-2aaf1d28-55c3-49e9-b981-91bc8f044685 service nova] Creating event network-vif-plugged:a44b7ed2-6364-44d9-9fbc-d45c00c0413e for instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f on hv-cinder-84336 May 28 03:36:41.499396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-5f43dd71-7518-4e0f-8b52-91303427fba1 req-2aaf1d28-55c3-49e9-b981-91bc8f044685 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029469 May 28 03:36:41.499983 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 84/146] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:41.704922 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-579ba76c-1316-4d20-93ca-07b3ae162db0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:41.704922 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-579ba76c-1316-4d20-93ca-07b3ae162db0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:41.708415 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-579ba76c-1316-4d20-93ca-07b3ae162db0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:41.709185 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-579ba76c-1316-4d20-93ca-07b3ae162db0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:41.710197 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-579ba76c-1316-4d20-93ca-07b3ae162db0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:41.710830 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-579ba76c-1316-4d20-93ca-07b3ae162db0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:41.859040 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-579ba76c-1316-4d20-93ca-07b3ae162db0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:41.894138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-579ba76c-1316-4d20-93ca-07b3ae162db0 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.196848 May 28 03:36:41.903285 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 63/147] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:41 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:42.002094 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-aadd8567-63de-4f6e-a547-fcdc6117a51a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:42.002856 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-aadd8567-63de-4f6e-a547-fcdc6117a51a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:42.009075 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-aadd8567-63de-4f6e-a547-fcdc6117a51a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:42.009865 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-aadd8567-63de-4f6e-a547-fcdc6117a51a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:42.010390 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-aadd8567-63de-4f6e-a547-fcdc6117a51a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:42.010887 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-aadd8567-63de-4f6e-a547-fcdc6117a51a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:42.171240 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-aadd8567-63de-4f6e-a547-fcdc6117a51a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:42.179083 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-aadd8567-63de-4f6e-a547-fcdc6117a51a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.181299 May 28 03:36:42.179651 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 85/148] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:41 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:42.230601 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2f8d7a2a-daed-45cb-9bcd-361a690857b8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:42.231243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2f8d7a2a-daed-45cb-9bcd-361a690857b8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:42.238104 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2f8d7a2a-daed-45cb-9bcd-361a690857b8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:42.238550 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2f8d7a2a-daed-45cb-9bcd-361a690857b8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:42.239548 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2f8d7a2a-daed-45cb-9bcd-361a690857b8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:42.239985 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2f8d7a2a-daed-45cb-9bcd-361a690857b8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:42.387882 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2f8d7a2a-daed-45cb-9bcd-361a690857b8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:42.397567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2f8d7a2a-daed-45cb-9bcd-361a690857b8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.170633 May 28 03:36:42.398299 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 64/149] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:42 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:42.911450 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3c545f3b-5ae3-447b-9894-993e5c5cc50f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:42.912164 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3c545f3b-5ae3-447b-9894-993e5c5cc50f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:42.919447 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c545f3b-5ae3-447b-9894-993e5c5cc50f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:42.919976 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c545f3b-5ae3-447b-9894-993e5c5cc50f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:42.920894 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c545f3b-5ae3-447b-9894-993e5c5cc50f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:42.921450 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c545f3b-5ae3-447b-9894-993e5c5cc50f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:43.068671 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3c545f3b-5ae3-447b-9894-993e5c5cc50f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:43.075724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3c545f3b-5ae3-447b-9894-993e5c5cc50f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.169254 May 28 03:36:43.076285 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 86/150] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:42 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:43.195076 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2eafe7d8-64a2-4087-943c-7ab727587090 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:43.195790 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2eafe7d8-64a2-4087-943c-7ab727587090 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:43.205458 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2eafe7d8-64a2-4087-943c-7ab727587090 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:43.205943 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2eafe7d8-64a2-4087-943c-7ab727587090 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:43.206764 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2eafe7d8-64a2-4087-943c-7ab727587090 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:43.207210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2eafe7d8-64a2-4087-943c-7ab727587090 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:43.365158 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2eafe7d8-64a2-4087-943c-7ab727587090 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:43.372523 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2eafe7d8-64a2-4087-943c-7ab727587090 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1366 microversion: 2.1 time: 0.181377 May 28 03:36:43.373086 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 65/151] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:43 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1366 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:43.419754 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1a7b57bf-d396-4cc3-8460-7d975ff31e75 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:43.420366 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1a7b57bf-d396-4cc3-8460-7d975ff31e75 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:43.426716 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1a7b57bf-d396-4cc3-8460-7d975ff31e75 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:43.427191 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1a7b57bf-d396-4cc3-8460-7d975ff31e75 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:43.428197 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1a7b57bf-d396-4cc3-8460-7d975ff31e75 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:43.428699 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1a7b57bf-d396-4cc3-8460-7d975ff31e75 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:43.522412 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-b4fcd09e-bf86-4717-a671-80cdf92972bf req-5f1121be-1b47-44ab-8a15-d1575f612dad service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "247082dc-5284-462d-92c8-045154d864e4", "name": "network-vif-plugged", "status": "completed", "tag": "6aaa9bcc-6545-4d6b-95af-ee421ef5771b"}, {"server_uuid": "da053642-eac6-4484-9557-f0b878712cf2", "name": "network-vif-plugged", "status": "completed", "tag": "63e23069-fb2c-4804-8da1-bf97b736bc08"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:43.536572 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-b4fcd09e-bf86-4717-a671-80cdf92972bf req-5f1121be-1b47-44ab-8a15-d1575f612dad service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:43.537088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-b4fcd09e-bf86-4717-a671-80cdf92972bf req-5f1121be-1b47-44ab-8a15-d1575f612dad service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:43.558769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-b4fcd09e-bf86-4717-a671-80cdf92972bf req-5f1121be-1b47-44ab-8a15-d1575f612dad service nova] Creating event network-vif-plugged:6aaa9bcc-6545-4d6b-95af-ee421ef5771b for instance 247082dc-5284-462d-92c8-045154d864e4 on hv-cinder-84336 May 28 03:36:43.559379 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-b4fcd09e-bf86-4717-a671-80cdf92972bf req-5f1121be-1b47-44ab-8a15-d1575f612dad service nova] Creating event network-vif-plugged:63e23069-fb2c-4804-8da1-bf97b736bc08 for instance da053642-eac6-4484-9557-f0b878712cf2 on hv-cinder-84336 May 28 03:36:43.566080 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-b4fcd09e-bf86-4717-a671-80cdf92972bf req-5f1121be-1b47-44ab-8a15-d1575f612dad service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.047235 May 28 03:36:43.566674 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 66/152] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:43 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:36:43.581489 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1a7b57bf-d396-4cc3-8460-7d975ff31e75 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:43.588982 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1a7b57bf-d396-4cc3-8460-7d975ff31e75 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.173100 May 28 03:36:43.589586 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 87/153] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:43 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:44.093670 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-338890c4-3224-427b-8e8f-afe0c8348bc4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:44.094445 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-338890c4-3224-427b-8e8f-afe0c8348bc4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:44.102348 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-338890c4-3224-427b-8e8f-afe0c8348bc4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:44.103096 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-338890c4-3224-427b-8e8f-afe0c8348bc4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:44.104116 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-338890c4-3224-427b-8e8f-afe0c8348bc4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:44.104556 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-338890c4-3224-427b-8e8f-afe0c8348bc4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:44.302793 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-338890c4-3224-427b-8e8f-afe0c8348bc4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:44.310424 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-338890c4-3224-427b-8e8f-afe0c8348bc4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.221431 May 28 03:36:44.320033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 67/154] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:44 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:44.389386 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6c1d0e72-88ba-4ea2-a1be-4f4bdaf80f9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:44.390985 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-6c1d0e72-88ba-4ea2-a1be-4f4bdaf80f9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:44.397646 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6c1d0e72-88ba-4ea2-a1be-4f4bdaf80f9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:44.398160 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6c1d0e72-88ba-4ea2-a1be-4f4bdaf80f9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:44.398997 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6c1d0e72-88ba-4ea2-a1be-4f4bdaf80f9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:44.399429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6c1d0e72-88ba-4ea2-a1be-4f4bdaf80f9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:44.569076 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6c1d0e72-88ba-4ea2-a1be-4f4bdaf80f9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:44.577328 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6c1d0e72-88ba-4ea2-a1be-4f4bdaf80f9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1555 microversion: 2.1 time: 0.192099 May 28 03:36:44.578352 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 88/155] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:44 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1555 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:44.594384 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d57cfbed-937f-4847-84e8-fbfe673adac1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:44.594911 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d57cfbed-937f-4847-84e8-fbfe673adac1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:44.601781 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d57cfbed-937f-4847-84e8-fbfe673adac1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:44.602563 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d57cfbed-937f-4847-84e8-fbfe673adac1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:44.603518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d57cfbed-937f-4847-84e8-fbfe673adac1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:44.604110 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d57cfbed-937f-4847-84e8-fbfe673adac1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:44.607396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f744ab95-8feb-4a76-a710-7f18d00e352a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:44.608268 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f744ab95-8feb-4a76-a710-7f18d00e352a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:44.615856 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f744ab95-8feb-4a76-a710-7f18d00e352a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:44.616589 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f744ab95-8feb-4a76-a710-7f18d00e352a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:44.617402 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f744ab95-8feb-4a76-a710-7f18d00e352a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:44.618070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f744ab95-8feb-4a76-a710-7f18d00e352a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:44.742974 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d57cfbed-937f-4847-84e8-fbfe673adac1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.147612 May 28 03:36:44.747873 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 68/156] 10.222.0.41 () {58 vars in 1274 bytes} [Sat May 28 03:36:44 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-interface => generated 284 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:44.793290 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f744ab95-8feb-4a76-a710-7f18d00e352a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:44.814112 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f744ab95-8feb-4a76-a710-7f18d00e352a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.211574 May 28 03:36:44.815415 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 89/157] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:44 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:36:45.327606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-543e6beb-583b-47b5-8107-ad2d8fea5271 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:45.329562 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-543e6beb-583b-47b5-8107-ad2d8fea5271 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:45.338033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-543e6beb-583b-47b5-8107-ad2d8fea5271 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:45.338742 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-543e6beb-583b-47b5-8107-ad2d8fea5271 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:45.339755 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-543e6beb-583b-47b5-8107-ad2d8fea5271 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:45.340520 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-543e6beb-583b-47b5-8107-ad2d8fea5271 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:45.628875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-543e6beb-583b-47b5-8107-ad2d8fea5271 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:45.636825 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-543e6beb-583b-47b5-8107-ad2d8fea5271 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1363 microversion: 2.1 time: 0.314341 May 28 03:36:45.641173 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 69/158] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:45 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1363 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:45.836033 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a4e6b970-af45-43a0-b2bf-07e73048da85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:45.837296 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a4e6b970-af45-43a0-b2bf-07e73048da85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:45.848529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a4e6b970-af45-43a0-b2bf-07e73048da85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:45.848987 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a4e6b970-af45-43a0-b2bf-07e73048da85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:45.850056 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a4e6b970-af45-43a0-b2bf-07e73048da85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:45.850504 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a4e6b970-af45-43a0-b2bf-07e73048da85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:46.011938 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a4e6b970-af45-43a0-b2bf-07e73048da85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:46.019452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a4e6b970-af45-43a0-b2bf-07e73048da85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1322 microversion: 2.1 time: 0.188584 May 28 03:36:46.020543 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 90/159] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:45 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1322 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:46.268816 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-f777c2ae-6004-4cde-9504-b067cc4eff40 req-a126a16f-d5c1-4ef5-b9b3-d9bb404f8234 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "161bde4a-7b5a-4c34-9cad-add4855d9d1f", "tag": "a44b7ed2-6364-44d9-9fbc-d45c00c0413e"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:46.277212 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-f777c2ae-6004-4cde-9504-b067cc4eff40 req-a126a16f-d5c1-4ef5-b9b3-d9bb404f8234 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:46.278120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-f777c2ae-6004-4cde-9504-b067cc4eff40 req-a126a16f-d5c1-4ef5-b9b3-d9bb404f8234 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:46.292244 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-f777c2ae-6004-4cde-9504-b067cc4eff40 req-a126a16f-d5c1-4ef5-b9b3-d9bb404f8234 service nova] Creating event network-changed:a44b7ed2-6364-44d9-9fbc-d45c00c0413e for instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f on hv-cinder-84336 May 28 03:36:46.297475 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-f777c2ae-6004-4cde-9504-b067cc4eff40 req-a126a16f-d5c1-4ef5-b9b3-d9bb404f8234 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032661 May 28 03:36:46.298070 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 70/160] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:46.654471 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-73a70923-bbfb-47ce-88cc-81ec3aa8a8b1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:46.659381 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-73a70923-bbfb-47ce-88cc-81ec3aa8a8b1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:46.666893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-73a70923-bbfb-47ce-88cc-81ec3aa8a8b1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:46.667340 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-73a70923-bbfb-47ce-88cc-81ec3aa8a8b1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:46.669081 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-73a70923-bbfb-47ce-88cc-81ec3aa8a8b1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:46.669988 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-73a70923-bbfb-47ce-88cc-81ec3aa8a8b1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:46.696232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:46.850159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-73a70923-bbfb-47ce-88cc-81ec3aa8a8b1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:46.858477 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-73a70923-bbfb-47ce-88cc-81ec3aa8a8b1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1549 microversion: 2.1 time: 0.208455 May 28 03:36:46.859054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 91/161] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:46 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1549 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:46.877080 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cdf086de-49de-49e0-864d-243c21b07609 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:46.877667 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-cdf086de-49de-49e0-864d-243c21b07609 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:46.884463 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cdf086de-49de-49e0-864d-243c21b07609 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:46.884908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cdf086de-49de-49e0-864d-243c21b07609 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:46.885697 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cdf086de-49de-49e0-864d-243c21b07609 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:46.886127 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cdf086de-49de-49e0-864d-243c21b07609 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:46.980594 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-cdf086de-49de-49e0-864d-243c21b07609 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.107740 May 28 03:36:46.981213 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 71/162] 10.222.0.41 () {58 vars in 1274 bytes} [Sat May 28 03:36:46 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/os-interface => generated 284 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:47.037392 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bb46343c-b0dd-4ad5-b3d4-46ba59ff49a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:47.038176 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bb46343c-b0dd-4ad5-b3d4-46ba59ff49a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:47.069459 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bb46343c-b0dd-4ad5-b3d4-46ba59ff49a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:47.070101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bb46343c-b0dd-4ad5-b3d4-46ba59ff49a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:47.075533 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bb46343c-b0dd-4ad5-b3d4-46ba59ff49a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:47.075533 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bb46343c-b0dd-4ad5-b3d4-46ba59ff49a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:47.600984 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bb46343c-b0dd-4ad5-b3d4-46ba59ff49a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:47.610201 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bb46343c-b0dd-4ad5-b3d4-46ba59ff49a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1517 microversion: 2.1 time: 0.577539 May 28 03:36:47.616587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 92/163] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:47 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1517 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:47.628518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-db963b11-a5fa-45fb-9e23-0e8f4a51cdd9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:47.631074 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-db963b11-a5fa-45fb-9e23-0e8f4a51cdd9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:47.638981 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-db963b11-a5fa-45fb-9e23-0e8f4a51cdd9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:47.639436 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-db963b11-a5fa-45fb-9e23-0e8f4a51cdd9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:47.640219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-db963b11-a5fa-45fb-9e23-0e8f4a51cdd9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:47.640652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-db963b11-a5fa-45fb-9e23-0e8f4a51cdd9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:47.697308 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-db963b11-a5fa-45fb-9e23-0e8f4a51cdd9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lazy-loading 'flavor' on Instance uuid 247082dc-5284-462d-92c8-045154d864e4 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:36:48.023592 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-db963b11-a5fa-45fb-9e23-0e8f4a51cdd9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "POST /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4/action" status: 200 len: 32 microversion: 2.1 time: 0.399371 May 28 03:36:48.029140 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 72/164] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:36:47 2022] POST /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4/action => generated 32 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 28 03:36:48.036665 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4e7dcbe4-e17a-497a-95ca-fd8ad7806262 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:48.037737 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4e7dcbe4-e17a-497a-95ca-fd8ad7806262 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:48.050788 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4e7dcbe4-e17a-497a-95ca-fd8ad7806262 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:48.051556 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4e7dcbe4-e17a-497a-95ca-fd8ad7806262 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:48.052550 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4e7dcbe4-e17a-497a-95ca-fd8ad7806262 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:48.053173 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4e7dcbe4-e17a-497a-95ca-fd8ad7806262 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:48.277433 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4e7dcbe4-e17a-497a-95ca-fd8ad7806262 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:48.287413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4e7dcbe4-e17a-497a-95ca-fd8ad7806262 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.255813 May 28 03:36:48.296173 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 93/165] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:48 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:48.753616 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-5393c173-4ff9-4550-a98e-1b5a72adde06 req-cee82197-c3f3-4366-b833-3e16ad5c2a21 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "da053642-eac6-4484-9557-f0b878712cf2", "tag": "63e23069-fb2c-4804-8da1-bf97b736bc08"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:48.768686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-5393c173-4ff9-4550-a98e-1b5a72adde06 req-cee82197-c3f3-4366-b833-3e16ad5c2a21 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:48.769458 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-5393c173-4ff9-4550-a98e-1b5a72adde06 req-cee82197-c3f3-4366-b833-3e16ad5c2a21 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:48.785144 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-5393c173-4ff9-4550-a98e-1b5a72adde06 req-cee82197-c3f3-4366-b833-3e16ad5c2a21 service nova] Creating event network-changed:63e23069-fb2c-4804-8da1-bf97b736bc08 for instance da053642-eac6-4484-9557-f0b878712cf2 on hv-cinder-84336 May 28 03:36:48.790729 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-5393c173-4ff9-4550-a98e-1b5a72adde06 req-cee82197-c3f3-4366-b833-3e16ad5c2a21 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041625 May 28 03:36:48.791881 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 73/166] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:49.304645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-af318ad9-ef9a-4cca-8a57-a7a7da51b8ca tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:49.305269 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-af318ad9-ef9a-4cca-8a57-a7a7da51b8ca tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:49.317178 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af318ad9-ef9a-4cca-8a57-a7a7da51b8ca tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:49.319102 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af318ad9-ef9a-4cca-8a57-a7a7da51b8ca tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:49.321417 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af318ad9-ef9a-4cca-8a57-a7a7da51b8ca tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:49.323115 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af318ad9-ef9a-4cca-8a57-a7a7da51b8ca tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:49.581088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-af318ad9-ef9a-4cca-8a57-a7a7da51b8ca tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:49.589995 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-af318ad9-ef9a-4cca-8a57-a7a7da51b8ca tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.291084 May 28 03:36:49.591184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 94/167] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:49 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:50.828965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1a490554-c0f4-4798-a2d1-2ad806f2d586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:50.828965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1a490554-c0f4-4798-a2d1-2ad806f2d586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:50.828965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1a490554-c0f4-4798-a2d1-2ad806f2d586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:50.828965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1a490554-c0f4-4798-a2d1-2ad806f2d586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:50.832056 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1a490554-c0f4-4798-a2d1-2ad806f2d586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:50.832056 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1a490554-c0f4-4798-a2d1-2ad806f2d586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:50.832475 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-633f7c54-6e07-42bc-9bc4-c2a480e45c51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:50.832475 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-633f7c54-6e07-42bc-9bc4-c2a480e45c51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:50.832475 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-633f7c54-6e07-42bc-9bc4-c2a480e45c51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:50.832475 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-633f7c54-6e07-42bc-9bc4-c2a480e45c51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:50.833260 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-633f7c54-6e07-42bc-9bc4-c2a480e45c51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:50.833260 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-633f7c54-6e07-42bc-9bc4-c2a480e45c51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:50.833423 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1a490554-c0f4-4798-a2d1-2ad806f2d586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:36:50.833511 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-633f7c54-6e07-42bc-9bc4-c2a480e45c51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:50.833511 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-633f7c54-6e07-42bc-9bc4-c2a480e45c51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.163371 May 28 03:36:50.833511 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 95/168] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:50 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:36:51.337998 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1a490554-c0f4-4798-a2d1-2ad806f2d586 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "DELETE /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 204 len: 0 microversion: 2.1 time: 0.748809 May 28 03:36:51.338596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 74/169] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:36:50 2022] DELETE /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 0 bytes in 750 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:36:51.347644 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c4857bb2-dfb7-42c0-bd61-9d0e25543f67 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:51.348140 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c4857bb2-dfb7-42c0-bd61-9d0e25543f67 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:51.354748 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4857bb2-dfb7-42c0-bd61-9d0e25543f67 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:51.355217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4857bb2-dfb7-42c0-bd61-9d0e25543f67 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:51.356004 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4857bb2-dfb7-42c0-bd61-9d0e25543f67 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:51.356487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4857bb2-dfb7-42c0-bd61-9d0e25543f67 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:51.500955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c4857bb2-dfb7-42c0-bd61-9d0e25543f67 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:51.517557 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c4857bb2-dfb7-42c0-bd61-9d0e25543f67 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1552 microversion: 2.1 time: 0.166006 May 28 03:36:51.517557 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 96/170] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:51 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1552 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:51.848487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-42f41f89-7c04-4fb1-bea9-bfe34f72ac66 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:51.856555 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-42f41f89-7c04-4fb1-bea9-bfe34f72ac66 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:51.869124 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-42f41f89-7c04-4fb1-bea9-bfe34f72ac66 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:51.869586 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-42f41f89-7c04-4fb1-bea9-bfe34f72ac66 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:51.870372 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-42f41f89-7c04-4fb1-bea9-bfe34f72ac66 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:51.870811 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-42f41f89-7c04-4fb1-bea9-bfe34f72ac66 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:52.048189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-42f41f89-7c04-4fb1-bea9-bfe34f72ac66 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:52.053037 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-42f41f89-7c04-4fb1-bea9-bfe34f72ac66 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.211271 May 28 03:36:52.053650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 75/171] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:51 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:52.532281 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-56f0aa88-eba5-4d08-854f-25ab9f40935f tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:52.533258 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-56f0aa88-eba5-4d08-854f-25ab9f40935f tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:52.539787 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-56f0aa88-eba5-4d08-854f-25ab9f40935f tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:52.540239 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-56f0aa88-eba5-4d08-854f-25ab9f40935f tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:52.541009 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-56f0aa88-eba5-4d08-854f-25ab9f40935f tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:52.541481 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-56f0aa88-eba5-4d08-854f-25ab9f40935f tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:52.695798 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-56f0aa88-eba5-4d08-854f-25ab9f40935f tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:52.703756 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-56f0aa88-eba5-4d08-854f-25ab9f40935f tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1552 microversion: 2.1 time: 0.175388 May 28 03:36:52.704323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 97/172] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:52 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1552 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:53.070430 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-daff6736-4c95-41e7-803b-76b166bc5dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:53.071104 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-daff6736-4c95-41e7-803b-76b166bc5dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:53.078875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-daff6736-4c95-41e7-803b-76b166bc5dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:53.079764 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-daff6736-4c95-41e7-803b-76b166bc5dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:53.081550 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-daff6736-4c95-41e7-803b-76b166bc5dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:53.082066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-daff6736-4c95-41e7-803b-76b166bc5dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:53.222417 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-daff6736-4c95-41e7-803b-76b166bc5dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:53.229955 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-daff6736-4c95-41e7-803b-76b166bc5dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.163653 May 28 03:36:53.230522 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 76/173] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:53 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:53.720808 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9d562670-e339-4217-a0f3-9574446d62cb tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:53.721444 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9d562670-e339-4217-a0f3-9574446d62cb tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:53.728119 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9d562670-e339-4217-a0f3-9574446d62cb tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:53.728974 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9d562670-e339-4217-a0f3-9574446d62cb tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:53.730674 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9d562670-e339-4217-a0f3-9574446d62cb tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:53.730674 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9d562670-e339-4217-a0f3-9574446d62cb tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:53.922725 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9d562670-e339-4217-a0f3-9574446d62cb tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:53.932827 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9d562670-e339-4217-a0f3-9574446d62cb tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1552 microversion: 2.1 time: 0.216733 May 28 03:36:53.934133 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 98/174] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:53 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1552 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:54.246855 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ed237ff5-b9e8-4ee8-9cb9-bc560a396430 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:54.248050 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ed237ff5-b9e8-4ee8-9cb9-bc560a396430 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:54.255470 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ed237ff5-b9e8-4ee8-9cb9-bc560a396430 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:54.255913 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ed237ff5-b9e8-4ee8-9cb9-bc560a396430 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:54.256674 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ed237ff5-b9e8-4ee8-9cb9-bc560a396430 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:54.257326 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ed237ff5-b9e8-4ee8-9cb9-bc560a396430 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:54.261491 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:54.493488 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ed237ff5-b9e8-4ee8-9cb9-bc560a396430 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:54.500550 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ed237ff5-b9e8-4ee8-9cb9-bc560a396430 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.257491 May 28 03:36:54.501611 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 77/175] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:54 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:54.750352 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-59f04d97-5fe9-44e3-ae93-2a885422a1ca req-1604f6df-d301-4256-a602-1d639b713033 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "362687d7-ef88-4e49-b7a5-d732c28e1b0b", "name": "network-vif-deleted", "tag": "cda792d3-ee73-4947-a077-2caf88494a15"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:36:54.763470 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-59f04d97-5fe9-44e3-ae93-2a885422a1ca req-1604f6df-d301-4256-a602-1d639b713033 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:54.764384 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-59f04d97-5fe9-44e3-ae93-2a885422a1ca req-1604f6df-d301-4256-a602-1d639b713033 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:54.782630 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-59f04d97-5fe9-44e3-ae93-2a885422a1ca req-1604f6df-d301-4256-a602-1d639b713033 service nova] Creating event network-vif-deleted:cda792d3-ee73-4947-a077-2caf88494a15 for instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b on hv-cinder-84336 May 28 03:36:54.787815 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-59f04d97-5fe9-44e3-ae93-2a885422a1ca req-1604f6df-d301-4256-a602-1d639b713033 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041537 May 28 03:36:54.788421 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 99/176] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:36:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:36:54.951078 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a8c9458c-7780-4dd2-b819-588ec83f55e2 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:54.951946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a8c9458c-7780-4dd2-b819-588ec83f55e2 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:54.958776 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a8c9458c-7780-4dd2-b819-588ec83f55e2 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:54.962840 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a8c9458c-7780-4dd2-b819-588ec83f55e2 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:54.962840 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a8c9458c-7780-4dd2-b819-588ec83f55e2 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:54.962840 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a8c9458c-7780-4dd2-b819-588ec83f55e2 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:55.023399 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a8c9458c-7780-4dd2-b819-588ec83f55e2 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:55.031428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a8c9458c-7780-4dd2-b819-588ec83f55e2 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1345 microversion: 2.1 time: 0.084249 May 28 03:36:55.031996 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 78/177] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:54 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1345 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:55.521055 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e7db13a8-985d-47d8-83bb-4acdcf737559 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:55.521773 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e7db13a8-985d-47d8-83bb-4acdcf737559 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:55.533924 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e7db13a8-985d-47d8-83bb-4acdcf737559 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:55.534440 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e7db13a8-985d-47d8-83bb-4acdcf737559 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:55.535219 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e7db13a8-985d-47d8-83bb-4acdcf737559 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:55.535627 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e7db13a8-985d-47d8-83bb-4acdcf737559 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:55.688952 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e7db13a8-985d-47d8-83bb-4acdcf737559 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:55.698203 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e7db13a8-985d-47d8-83bb-4acdcf737559 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.182055 May 28 03:36:55.698829 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 100/178] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:55 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:56.053584 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3b7cd5e9-3fa2-4a7c-ad4f-57ffd39185a6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:56.053584 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3b7cd5e9-3fa2-4a7c-ad4f-57ffd39185a6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:56.062127 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3b7cd5e9-3fa2-4a7c-ad4f-57ffd39185a6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:56.062562 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3b7cd5e9-3fa2-4a7c-ad4f-57ffd39185a6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:56.063340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3b7cd5e9-3fa2-4a7c-ad4f-57ffd39185a6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:56.063776 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3b7cd5e9-3fa2-4a7c-ad4f-57ffd39185a6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:56.098403 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3b7cd5e9-3fa2-4a7c-ad4f-57ffd39185a6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:56.099037 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3b7cd5e9-3fa2-4a7c-ad4f-57ffd39185a6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:56.100113 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-3b7cd5e9-3fa2-4a7c-ad4f-57ffd39185a6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lazy-loading 'fault' on Instance uuid 362687d7-ef88-4e49-b7a5-d732c28e1b0b {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:36:56.168865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3b7cd5e9-3fa2-4a7c-ad4f-57ffd39185a6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46e48b861bc2498c9670de030c783f54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:56.185978 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3b7cd5e9-3fa2-4a7c-ad4f-57ffd39185a6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 200 len: 1350 microversion: 2.1 time: 0.133424 May 28 03:36:56.186629 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 79/179] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:56 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 1350 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:56.715755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-673e2b2a-6ab6-4842-9376-b52604754b0a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:56.717235 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-673e2b2a-6ab6-4842-9376-b52604754b0a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:56.724186 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-673e2b2a-6ab6-4842-9376-b52604754b0a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:56.728250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-673e2b2a-6ab6-4842-9376-b52604754b0a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:56.728250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-673e2b2a-6ab6-4842-9376-b52604754b0a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:56.728250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-673e2b2a-6ab6-4842-9376-b52604754b0a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:56.869126 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-673e2b2a-6ab6-4842-9376-b52604754b0a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:56.876816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-673e2b2a-6ab6-4842-9376-b52604754b0a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.164913 May 28 03:36:56.877450 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 101/180] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:56 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:57.204541 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-dfab423d-a7cb-40b5-bbeb-d70c7674b6b6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:57.205189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-dfab423d-a7cb-40b5-bbeb-d70c7674b6b6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:57.211781 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dfab423d-a7cb-40b5-bbeb-d70c7674b6b6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:57.212224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dfab423d-a7cb-40b5-bbeb-d70c7674b6b6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:57.212971 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dfab423d-a7cb-40b5-bbeb-d70c7674b6b6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:57.213449 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dfab423d-a7cb-40b5-bbeb-d70c7674b6b6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:57.228285 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-dfab423d-a7cb-40b5-bbeb-d70c7674b6b6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] HTTP exception thrown: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-dfab423d-a7cb-40b5-bbeb-d70c7674b6b6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Request method failure captured: May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b HTTP/1.1 May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZiVvo4phYUp3LeMz0tbg-qLj28jShV_sNp5MonwxhJnKqzCBWUftU2y84S1yggI5oaE9W2DSjaWB59OU8thCae1dtkqgNPAgWHxO-5_DIE5zz2Bmvlhs7lq2q5EZJ_DGqF-KisoGlR9WCVOGDg_hY-8XzsDgavRXgOajub8Mf7SYWY May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: fc3ad7416d27411cb8ddbce0a3da18eb May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:36:57.232853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: fc3ad7416d27411cb8ddbce0a3da18eb May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-DeleteServersTestJSON-2145387111-project May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 46e48b861bc2498c9670de030c783f54 May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-DeleteServersTestJSON-2145387111-project May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:57.234338 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '362687d7-ef88-4e49-b7a5-d732c28e1b0b'} May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:36:57.235190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:57.236940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:36:57.238710 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-dfab423d-a7cb-40b5-bbeb-d70c7674b6b6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Returning 404 to user: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:36:57.238710 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-dfab423d-a7cb-40b5-bbeb-d70c7674b6b6 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 404 len: 111 microversion: 2.1 time: 0.033647 May 28 03:36:57.238710 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 80/181] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:57 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:36:57.378798 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f74166b7-d99c-4e3d-8e50-3e1b37298fc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:57.380202 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f74166b7-d99c-4e3d-8e50-3e1b37298fc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:57.387457 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f74166b7-d99c-4e3d-8e50-3e1b37298fc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:57.388208 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f74166b7-d99c-4e3d-8e50-3e1b37298fc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:57.389161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f74166b7-d99c-4e3d-8e50-3e1b37298fc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:57.389892 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f74166b7-d99c-4e3d-8e50-3e1b37298fc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:57.438006 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-f74166b7-d99c-4e3d-8e50-3e1b37298fc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] HTTP exception thrown: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f74166b7-d99c-4e3d-8e50-3e1b37298fc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Request method failure captured: May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b/os-volume_attachments HTTP/1.1 May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZiVvo4phYUp3LeMz0tbg-qLj28jShV_sNp5MonwxhJnKqzCBWUftU2y84S1yggI5oaE9W2DSjaWB59OU8thCae1dtkqgNPAgWHxO-5_DIE5zz2Bmvlhs7lq2q5EZJ_DGqF-KisoGlR9WCVOGDg_hY-8XzsDgavRXgOajub8Mf7SYWY May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: fc3ad7416d27411cb8ddbce0a3da18eb May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:36:57.452755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: fc3ad7416d27411cb8ddbce0a3da18eb May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-DeleteServersTestJSON-2145387111-project May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 46e48b861bc2498c9670de030c783f54 May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-DeleteServersTestJSON-2145387111-project May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:57.454664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'server_id': '362687d7-ef88-4e49-b7a5-d732c28e1b0b'} May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:36:57.455462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:57.457333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:36:57.458621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f74166b7-d99c-4e3d-8e50-3e1b37298fc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Returning 404 to user: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:36:57.458621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f74166b7-d99c-4e3d-8e50-3e1b37298fc9 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.071350 May 28 03:36:57.458621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 102/182] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:36:57 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b/os-volume_attachments => generated 111 bytes in 73 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:36:57.894752 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-31ae8309-5a79-4c5a-a304-b9c45437d31f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:57.906837 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-31ae8309-5a79-4c5a-a304-b9c45437d31f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:57.906837 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-31ae8309-5a79-4c5a-a304-b9c45437d31f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:57.906837 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-31ae8309-5a79-4c5a-a304-b9c45437d31f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:57.906837 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-31ae8309-5a79-4c5a-a304-b9c45437d31f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:57.907862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-31ae8309-5a79-4c5a-a304-b9c45437d31f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:58.046523 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-31ae8309-5a79-4c5a-a304-b9c45437d31f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:58.054141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-31ae8309-5a79-4c5a-a304-b9c45437d31f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.163866 May 28 03:36:58.054709 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 81/183] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:57 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:36:58.942161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dd8d6cc9-e43c-4053-b41b-9654522fdf37 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:58.942753 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dd8d6cc9-e43c-4053-b41b-9654522fdf37 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:58.949353 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd8d6cc9-e43c-4053-b41b-9654522fdf37 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:58.950054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd8d6cc9-e43c-4053-b41b-9654522fdf37 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:58.950949 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd8d6cc9-e43c-4053-b41b-9654522fdf37 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:58.951486 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd8d6cc9-e43c-4053-b41b-9654522fdf37 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:58.965806 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-dd8d6cc9-e43c-4053-b41b-9654522fdf37 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] HTTP exception thrown: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dd8d6cc9-e43c-4053-b41b-9654522fdf37 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Request method failure captured: May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b HTTP/1.1 May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZiVvo4phYUp3LeMz0tbg-qLj28jShV_sNp5MonwxhJnKqzCBWUftU2y84S1yggI5oaE9W2DSjaWB59OU8thCae1dtkqgNPAgWHxO-5_DIE5zz2Bmvlhs7lq2q5EZJ_DGqF-KisoGlR9WCVOGDg_hY-8XzsDgavRXgOajub8Mf7SYWY May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: fc3ad7416d27411cb8ddbce0a3da18eb May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:36:58.968166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: fc3ad7416d27411cb8ddbce0a3da18eb May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-DeleteServersTestJSON-2145387111-project May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 46e48b861bc2498c9670de030c783f54 May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-DeleteServersTestJSON-2145387111-project May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:58.969540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': '362687d7-ef88-4e49-b7a5-d732c28e1b0b'} May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:36:58.970342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:58.972123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:36:58.979009 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dd8d6cc9-e43c-4053-b41b-9654522fdf37 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Returning 404 to user: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:36:58.979009 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-dd8d6cc9-e43c-4053-b41b-9654522fdf37 tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "DELETE /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 404 len: 111 microversion: 2.1 time: 0.036662 May 28 03:36:58.979009 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 103/184] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:36:58 2022] DELETE /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:36:58.985776 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-edb7c1fc-f678-4790-9910-a433aa14732a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:58.986390 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-edb7c1fc-f678-4790-9910-a433aa14732a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] [instance: 362687d7-ef88-4e49-b7a5-d732c28e1b0b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:58.994188 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-edb7c1fc-f678-4790-9910-a433aa14732a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:58.994638 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-edb7c1fc-f678-4790-9910-a433aa14732a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:58.995414 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-edb7c1fc-f678-4790-9910-a433aa14732a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:58.995845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-edb7c1fc-f678-4790-9910-a433aa14732a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:59.010167 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-edb7c1fc-f678-4790-9910-a433aa14732a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] HTTP exception thrown: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-edb7c1fc-f678-4790-9910-a433aa14732a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Request method failure captured: May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b HTTP/1.1 May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZiVvo4phYUp3LeMz0tbg-qLj28jShV_sNp5MonwxhJnKqzCBWUftU2y84S1yggI5oaE9W2DSjaWB59OU8thCae1dtkqgNPAgWHxO-5_DIE5zz2Bmvlhs7lq2q5EZJ_DGqF-KisoGlR9WCVOGDg_hY-8XzsDgavRXgOajub8Mf7SYWY May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: fc3ad7416d27411cb8ddbce0a3da18eb May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:36:59.011682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_fc3ad7416d27411cb8ddbce0a3da18eb"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: fc3ad7416d27411cb8ddbce0a3da18eb May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-DeleteServersTestJSON-2145387111 May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-DeleteServersTestJSON-2145387111-project May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 46e48b861bc2498c9670de030c783f54 May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-DeleteServersTestJSON-2145387111-project May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:59.013150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '362687d7-ef88-4e49-b7a5-d732c28e1b0b'} May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:36:59.014182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. May 28 03:36:59.018786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:36:59.020352 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-edb7c1fc-f678-4790-9910-a433aa14732a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] Returning 404 to user: Instance 362687d7-ef88-4e49-b7a5-d732c28e1b0b could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:36:59.020352 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-edb7c1fc-f678-4790-9910-a433aa14732a tempest-DeleteServersTestJSON-2145387111 tempest-DeleteServersTestJSON-2145387111-project] 10.222.0.41 "GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b" status: 404 len: 111 microversion: 2.1 time: 0.032750 May 28 03:36:59.020352 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 82/185] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:58 2022] GET /compute/v2.1/servers/362687d7-ef88-4e49-b7a5-d732c28e1b0b => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:36:59.073365 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c3e45d86-70bd-47ec-b1eb-e4ae18daf5ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:36:59.073986 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c3e45d86-70bd-47ec-b1eb-e4ae18daf5ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:36:59.080373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c3e45d86-70bd-47ec-b1eb-e4ae18daf5ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:59.081436 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c3e45d86-70bd-47ec-b1eb-e4ae18daf5ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:59.082228 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c3e45d86-70bd-47ec-b1eb-e4ae18daf5ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:36:59.082868 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c3e45d86-70bd-47ec-b1eb-e4ae18daf5ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:36:59.482261 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c3e45d86-70bd-47ec-b1eb-e4ae18daf5ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:36:59.494705 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c3e45d86-70bd-47ec-b1eb-e4ae18daf5ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.425191 May 28 03:36:59.495367 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 104/186] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:36:59 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:00.512269 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0a48e2ac-4335-4175-bd95-6fd1555c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:00.512799 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0a48e2ac-4335-4175-bd95-6fd1555c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:00.519935 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0a48e2ac-4335-4175-bd95-6fd1555c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:00.520379 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0a48e2ac-4335-4175-bd95-6fd1555c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:00.521175 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0a48e2ac-4335-4175-bd95-6fd1555c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:00.521606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0a48e2ac-4335-4175-bd95-6fd1555c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:00.664633 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0a48e2ac-4335-4175-bd95-6fd1555c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:00.672511 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0a48e2ac-4335-4175-bd95-6fd1555c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.164525 May 28 03:37:00.673072 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 83/187] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:00 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:01.692126 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-68571520-6e89-411f-9271-9cf8d00b38d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:01.692728 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-68571520-6e89-411f-9271-9cf8d00b38d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:01.700235 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68571520-6e89-411f-9271-9cf8d00b38d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:01.701054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68571520-6e89-411f-9271-9cf8d00b38d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:01.702161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68571520-6e89-411f-9271-9cf8d00b38d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:01.702761 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68571520-6e89-411f-9271-9cf8d00b38d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:01.847356 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-68571520-6e89-411f-9271-9cf8d00b38d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:01.855776 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-68571520-6e89-411f-9271-9cf8d00b38d9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.168013 May 28 03:37:01.856354 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 105/188] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:01 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:02.874036 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5d28e933-3827-48f6-a9ad-2eaeb750f499 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:02.874976 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5d28e933-3827-48f6-a9ad-2eaeb750f499 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:02.884265 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5d28e933-3827-48f6-a9ad-2eaeb750f499 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:02.884768 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5d28e933-3827-48f6-a9ad-2eaeb750f499 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:02.885599 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5d28e933-3827-48f6-a9ad-2eaeb750f499 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:02.886226 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5d28e933-3827-48f6-a9ad-2eaeb750f499 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:03.037693 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5d28e933-3827-48f6-a9ad-2eaeb750f499 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:03.045513 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5d28e933-3827-48f6-a9ad-2eaeb750f499 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.175723 May 28 03:37:03.046446 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 84/189] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:02 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:04.064185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1114be87-3adc-4fb9-8608-1960d89275ee tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:04.066822 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1114be87-3adc-4fb9-8608-1960d89275ee tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:04.079423 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1114be87-3adc-4fb9-8608-1960d89275ee tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:04.082331 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1114be87-3adc-4fb9-8608-1960d89275ee tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:04.083139 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1114be87-3adc-4fb9-8608-1960d89275ee tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:04.083583 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1114be87-3adc-4fb9-8608-1960d89275ee tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:04.235539 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1114be87-3adc-4fb9-8608-1960d89275ee tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:04.246452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1114be87-3adc-4fb9-8608-1960d89275ee tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.185107 May 28 03:37:04.246452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 106/190] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:04 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:05.264207 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f39e586f-2316-4c88-93e5-0769addb6877 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:05.264207 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f39e586f-2316-4c88-93e5-0769addb6877 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:05.267394 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=68,nova_api:UPDATE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:05.271011 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f39e586f-2316-4c88-93e5-0769addb6877 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:05.271465 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f39e586f-2316-4c88-93e5-0769addb6877 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:05.272241 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f39e586f-2316-4c88-93e5-0769addb6877 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:05.272681 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f39e586f-2316-4c88-93e5-0769addb6877 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:05.488166 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f39e586f-2316-4c88-93e5-0769addb6877 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:05.496009 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f39e586f-2316-4c88-93e5-0769addb6877 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.237648 May 28 03:37:05.496581 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 85/191] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:05 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:06.515107 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b8aed810-9688-4d2a-a966-04cbf8b8ccaf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:06.516638 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b8aed810-9688-4d2a-a966-04cbf8b8ccaf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:06.533560 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b8aed810-9688-4d2a-a966-04cbf8b8ccaf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:06.534913 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b8aed810-9688-4d2a-a966-04cbf8b8ccaf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:06.535713 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b8aed810-9688-4d2a-a966-04cbf8b8ccaf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:06.536332 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b8aed810-9688-4d2a-a966-04cbf8b8ccaf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:06.701622 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b8aed810-9688-4d2a-a966-04cbf8b8ccaf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:06.717901 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b8aed810-9688-4d2a-a966-04cbf8b8ccaf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.207455 May 28 03:37:06.719342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 107/192] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:06 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:07.745063 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1ec646b9-0026-41f3-b00d-50f7ea8b423b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:07.745063 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1ec646b9-0026-41f3-b00d-50f7ea8b423b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:07.745063 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ec646b9-0026-41f3-b00d-50f7ea8b423b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:07.745063 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ec646b9-0026-41f3-b00d-50f7ea8b423b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:07.749493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ec646b9-0026-41f3-b00d-50f7ea8b423b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:07.750141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ec646b9-0026-41f3-b00d-50f7ea8b423b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:07.901475 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1ec646b9-0026-41f3-b00d-50f7ea8b423b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:07.910234 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1ec646b9-0026-41f3-b00d-50f7ea8b423b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.178332 May 28 03:37:07.910811 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 86/193] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:07 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:08.928330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e149c270-3ebd-452e-b7cc-2238a410762e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:08.929009 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e149c270-3ebd-452e-b7cc-2238a410762e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:08.931050 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=68 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:08.936218 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e149c270-3ebd-452e-b7cc-2238a410762e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:08.936862 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e149c270-3ebd-452e-b7cc-2238a410762e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:08.937953 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e149c270-3ebd-452e-b7cc-2238a410762e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:08.938452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e149c270-3ebd-452e-b7cc-2238a410762e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:09.078667 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e149c270-3ebd-452e-b7cc-2238a410762e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:09.086633 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e149c270-3ebd-452e-b7cc-2238a410762e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.162556 May 28 03:37:09.087226 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 108/194] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:08 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:10.104344 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0108141a-cc66-4370-8a8c-c94fd6746128 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:10.105131 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0108141a-cc66-4370-8a8c-c94fd6746128 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:10.112324 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0108141a-cc66-4370-8a8c-c94fd6746128 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:10.113045 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0108141a-cc66-4370-8a8c-c94fd6746128 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:10.114018 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0108141a-cc66-4370-8a8c-c94fd6746128 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:10.114720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0108141a-cc66-4370-8a8c-c94fd6746128 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:10.255243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0108141a-cc66-4370-8a8c-c94fd6746128 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:10.262707 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0108141a-cc66-4370-8a8c-c94fd6746128 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.162245 May 28 03:37:10.263276 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 87/195] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:10 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:11.279532 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6412ccc3-cc38-456e-8b34-f5db0d2098e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:11.280140 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-6412ccc3-cc38-456e-8b34-f5db0d2098e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:11.286982 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6412ccc3-cc38-456e-8b34-f5db0d2098e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:11.287429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6412ccc3-cc38-456e-8b34-f5db0d2098e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:11.288188 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6412ccc3-cc38-456e-8b34-f5db0d2098e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:11.288623 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6412ccc3-cc38-456e-8b34-f5db0d2098e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:11.429633 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6412ccc3-cc38-456e-8b34-f5db0d2098e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:11.437470 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6412ccc3-cc38-456e-8b34-f5db0d2098e3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.161565 May 28 03:37:11.438046 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 109/196] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:11 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:12.454330 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cf468383-13c9-42d8-aed9-13861c0da9c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:12.454865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-cf468383-13c9-42d8-aed9-13861c0da9c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:12.461452 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cf468383-13c9-42d8-aed9-13861c0da9c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:12.461905 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cf468383-13c9-42d8-aed9-13861c0da9c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:12.462685 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cf468383-13c9-42d8-aed9-13861c0da9c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:12.463118 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cf468383-13c9-42d8-aed9-13861c0da9c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:12.600990 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-cf468383-13c9-42d8-aed9-13861c0da9c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:12.608498 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-cf468383-13c9-42d8-aed9-13861c0da9c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.158045 May 28 03:37:12.609069 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 88/197] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:12 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:13.626191 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-57ff00cf-8d37-446a-80ff-bd75763b996a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:13.626909 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-57ff00cf-8d37-446a-80ff-bd75763b996a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:13.633894 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-57ff00cf-8d37-446a-80ff-bd75763b996a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:13.634701 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-57ff00cf-8d37-446a-80ff-bd75763b996a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:13.635572 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-57ff00cf-8d37-446a-80ff-bd75763b996a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:13.636012 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-57ff00cf-8d37-446a-80ff-bd75763b996a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:13.640250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:13.764045 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-57ff00cf-8d37-446a-80ff-bd75763b996a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:13.771378 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-57ff00cf-8d37-446a-80ff-bd75763b996a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.148956 May 28 03:37:13.771974 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 110/198] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:13 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:14.788320 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-00c572fc-85b1-4dbc-9cb0-f37b992b3b67 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:14.788975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-00c572fc-85b1-4dbc-9cb0-f37b992b3b67 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:14.795433 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-00c572fc-85b1-4dbc-9cb0-f37b992b3b67 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:14.795878 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-00c572fc-85b1-4dbc-9cb0-f37b992b3b67 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:14.796636 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-00c572fc-85b1-4dbc-9cb0-f37b992b3b67 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:14.797122 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-00c572fc-85b1-4dbc-9cb0-f37b992b3b67 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:14.926103 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-00c572fc-85b1-4dbc-9cb0-f37b992b3b67 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:14.933656 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-00c572fc-85b1-4dbc-9cb0-f37b992b3b67 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.149016 May 28 03:37:14.934234 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 89/199] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:14 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:15.951231 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-306896ca-ee75-435a-a9f8-91dd3c574043 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:15.951858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-306896ca-ee75-435a-a9f8-91dd3c574043 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:15.958997 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-306896ca-ee75-435a-a9f8-91dd3c574043 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:15.959442 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-306896ca-ee75-435a-a9f8-91dd3c574043 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:15.960210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-306896ca-ee75-435a-a9f8-91dd3c574043 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:15.960645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-306896ca-ee75-435a-a9f8-91dd3c574043 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:16.091668 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-306896ca-ee75-435a-a9f8-91dd3c574043 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:16.099067 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-306896ca-ee75-435a-a9f8-91dd3c574043 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.152180 May 28 03:37:16.099628 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 111/200] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:15 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:17.115217 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a3894e0c-d9f4-4b8a-81c1-cab6196b3124 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:17.115825 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a3894e0c-d9f4-4b8a-81c1-cab6196b3124 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:17.122737 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3894e0c-d9f4-4b8a-81c1-cab6196b3124 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:17.123214 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3894e0c-d9f4-4b8a-81c1-cab6196b3124 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:17.124012 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3894e0c-d9f4-4b8a-81c1-cab6196b3124 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:17.124445 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3894e0c-d9f4-4b8a-81c1-cab6196b3124 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:17.258591 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a3894e0c-d9f4-4b8a-81c1-cab6196b3124 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:17.266100 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a3894e0c-d9f4-4b8a-81c1-cab6196b3124 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.154897 May 28 03:37:17.266665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 90/201] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:17 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:18.285473 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4c1872cb-7e73-4bd9-8c60-6ee1a2347c5a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:18.286129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4c1872cb-7e73-4bd9-8c60-6ee1a2347c5a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:18.292214 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c1872cb-7e73-4bd9-8c60-6ee1a2347c5a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:18.292740 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c1872cb-7e73-4bd9-8c60-6ee1a2347c5a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:18.293609 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c1872cb-7e73-4bd9-8c60-6ee1a2347c5a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:18.294048 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c1872cb-7e73-4bd9-8c60-6ee1a2347c5a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:18.424418 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4c1872cb-7e73-4bd9-8c60-6ee1a2347c5a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:18.432287 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4c1872cb-7e73-4bd9-8c60-6ee1a2347c5a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.152282 May 28 03:37:18.433222 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 112/202] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:18 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:19.449402 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-795eb3e7-4754-47b0-a26b-bb9be6b1de51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:19.450097 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-795eb3e7-4754-47b0-a26b-bb9be6b1de51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:19.456584 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-795eb3e7-4754-47b0-a26b-bb9be6b1de51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:19.457034 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-795eb3e7-4754-47b0-a26b-bb9be6b1de51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:19.457837 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-795eb3e7-4754-47b0-a26b-bb9be6b1de51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:19.458277 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-795eb3e7-4754-47b0-a26b-bb9be6b1de51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:19.602734 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-795eb3e7-4754-47b0-a26b-bb9be6b1de51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:19.610690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-795eb3e7-4754-47b0-a26b-bb9be6b1de51 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.165140 May 28 03:37:19.611254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 91/203] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:19 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:20.627772 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b00ab58f-48e2-4f3e-b329-f05381a3e1e8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:20.628995 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b00ab58f-48e2-4f3e-b329-f05381a3e1e8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:20.635528 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b00ab58f-48e2-4f3e-b329-f05381a3e1e8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:20.635970 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b00ab58f-48e2-4f3e-b329-f05381a3e1e8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:20.636733 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b00ab58f-48e2-4f3e-b329-f05381a3e1e8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:20.637192 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b00ab58f-48e2-4f3e-b329-f05381a3e1e8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:20.767346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b00ab58f-48e2-4f3e-b329-f05381a3e1e8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:20.774820 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b00ab58f-48e2-4f3e-b329-f05381a3e1e8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.151137 May 28 03:37:20.775383 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 113/204] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:20 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:21.797375 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e7815939-ece7-4096-ba92-4d64f0ba5b38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:21.802344 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e7815939-ece7-4096-ba92-4d64f0ba5b38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:21.805460 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7815939-ece7-4096-ba92-4d64f0ba5b38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:21.805912 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7815939-ece7-4096-ba92-4d64f0ba5b38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:21.806681 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7815939-ece7-4096-ba92-4d64f0ba5b38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:21.807115 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7815939-ece7-4096-ba92-4d64f0ba5b38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:21.939783 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e7815939-ece7-4096-ba92-4d64f0ba5b38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:21.947776 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e7815939-ece7-4096-ba92-4d64f0ba5b38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.159390 May 28 03:37:21.948337 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 92/205] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:21 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:22.964772 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c552e355-43de-4953-a271-fcb9798c6fd4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:22.967271 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c552e355-43de-4953-a271-fcb9798c6fd4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:22.972627 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c552e355-43de-4953-a271-fcb9798c6fd4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:22.973096 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c552e355-43de-4953-a271-fcb9798c6fd4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:22.973936 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c552e355-43de-4953-a271-fcb9798c6fd4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:22.974377 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c552e355-43de-4953-a271-fcb9798c6fd4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:23.104845 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c552e355-43de-4953-a271-fcb9798c6fd4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:23.112488 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c552e355-43de-4953-a271-fcb9798c6fd4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.151297 May 28 03:37:23.113199 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 114/206] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:22 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:24.129616 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6737794d-2222-4447-ac51-49e8302f6d73 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:24.130226 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6737794d-2222-4447-ac51-49e8302f6d73 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:24.136655 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6737794d-2222-4447-ac51-49e8302f6d73 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:24.137124 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6737794d-2222-4447-ac51-49e8302f6d73 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:24.137898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6737794d-2222-4447-ac51-49e8302f6d73 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:24.138331 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6737794d-2222-4447-ac51-49e8302f6d73 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:24.275097 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6737794d-2222-4447-ac51-49e8302f6d73 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:24.282492 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6737794d-2222-4447-ac51-49e8302f6d73 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.156806 May 28 03:37:24.283279 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 93/207] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:24 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:25.301272 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-109ff0d7-a0f1-4441-b818-f0410ec42b36 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:25.301909 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-109ff0d7-a0f1-4441-b818-f0410ec42b36 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:25.307891 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-109ff0d7-a0f1-4441-b818-f0410ec42b36 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:25.308343 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-109ff0d7-a0f1-4441-b818-f0410ec42b36 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:25.309123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-109ff0d7-a0f1-4441-b818-f0410ec42b36 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:25.309586 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-109ff0d7-a0f1-4441-b818-f0410ec42b36 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:25.466591 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-109ff0d7-a0f1-4441-b818-f0410ec42b36 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:25.479316 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-109ff0d7-a0f1-4441-b818-f0410ec42b36 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.182235 May 28 03:37:25.480422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 115/208] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:25 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:37:26.497995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9b99eaf4-f051-4942-b446-4f20f4532974 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:26.500341 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9b99eaf4-f051-4942-b446-4f20f4532974 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:26.505980 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9b99eaf4-f051-4942-b446-4f20f4532974 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:26.506428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9b99eaf4-f051-4942-b446-4f20f4532974 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:26.507198 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9b99eaf4-f051-4942-b446-4f20f4532974 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:26.507635 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9b99eaf4-f051-4942-b446-4f20f4532974 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:26.517637 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=96 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:26.686288 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9b99eaf4-f051-4942-b446-4f20f4532974 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:26.695865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9b99eaf4-f051-4942-b446-4f20f4532974 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.201633 May 28 03:37:26.700632 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 94/209] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:26 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:27.714181 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c49048d2-a86c-4900-bb9c-b78dff501247 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:27.714893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c49048d2-a86c-4900-bb9c-b78dff501247 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:27.721238 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c49048d2-a86c-4900-bb9c-b78dff501247 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:27.721811 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c49048d2-a86c-4900-bb9c-b78dff501247 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:27.722774 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c49048d2-a86c-4900-bb9c-b78dff501247 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:27.723280 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c49048d2-a86c-4900-bb9c-b78dff501247 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:27.857116 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c49048d2-a86c-4900-bb9c-b78dff501247 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:27.872299 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c49048d2-a86c-4900-bb9c-b78dff501247 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1523 microversion: 2.1 time: 0.162567 May 28 03:37:27.876944 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 116/210] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:27 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1523 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:28.891108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2d249c69-5677-43fe-b5f4-82df612f7147 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:28.891726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2d249c69-5677-43fe-b5f4-82df612f7147 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:28.898186 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2d249c69-5677-43fe-b5f4-82df612f7147 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:28.898641 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2d249c69-5677-43fe-b5f4-82df612f7147 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:28.899417 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2d249c69-5677-43fe-b5f4-82df612f7147 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:28.900364 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2d249c69-5677-43fe-b5f4-82df612f7147 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:29.055544 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2d249c69-5677-43fe-b5f4-82df612f7147 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:29.063166 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2d249c69-5677-43fe-b5f4-82df612f7147 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1503 microversion: 2.1 time: 0.176643 May 28 03:37:29.063738 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 95/211] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:28 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1503 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:29.088226 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8c44b127-0503-4f1c-9b31-1000961e0a2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:29.088828 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8c44b127-0503-4f1c-9b31-1000961e0a2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:29.095079 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8c44b127-0503-4f1c-9b31-1000961e0a2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:29.095523 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8c44b127-0503-4f1c-9b31-1000961e0a2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:29.096286 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8c44b127-0503-4f1c-9b31-1000961e0a2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:29.096719 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8c44b127-0503-4f1c-9b31-1000961e0a2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:29.228349 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8c44b127-0503-4f1c-9b31-1000961e0a2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:29.236089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8c44b127-0503-4f1c-9b31-1000961e0a2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1516 microversion: 2.1 time: 0.151674 May 28 03:37:29.236749 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 117/212] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:29 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1516 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:33.141629 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-dc604fee-7c40-4f32-97bf-b9b5e045b300 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:33.142432 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-dc604fee-7c40-4f32-97bf-b9b5e045b300 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:33.150038 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc604fee-7c40-4f32-97bf-b9b5e045b300 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:33.150487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc604fee-7c40-4f32-97bf-b9b5e045b300 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:33.151588 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc604fee-7c40-4f32-97bf-b9b5e045b300 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:33.152023 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc604fee-7c40-4f32-97bf-b9b5e045b300 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:33.177673 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-dc604fee-7c40-4f32-97bf-b9b5e045b300 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/ips" status: 200 len: 152 microversion: 2.20 time: 0.040425 May 28 03:37:33.187574 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 96/213] 10.222.0.41 () {60 vars in 1288 bytes} [Sat May 28 03:37:33 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/ips => generated 152 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:37:34.241539 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d429c8af-91c9-48b1-9bf9-dc810132ec62 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:37:34.243742 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d429c8af-91c9-48b1-9bf9-dc810132ec62 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:34.251937 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d429c8af-91c9-48b1-9bf9-dc810132ec62 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:34.252394 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d429c8af-91c9-48b1-9bf9-dc810132ec62 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:34.253825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d429c8af-91c9-48b1-9bf9-dc810132ec62 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:34.254217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d429c8af-91c9-48b1-9bf9-dc810132ec62 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:34.293897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-d429c8af-91c9-48b1-9bf9-dc810132ec62 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lazy-loading 'flavor' on Instance uuid 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:37:34.689406 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d429c8af-91c9-48b1-9bf9-dc810132ec62 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "POST /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8/action" status: 200 len: 32 microversion: 2.1 time: 0.454190 May 28 03:37:34.689942 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 118/214] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:37:34 2022] POST /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8/action => generated 32 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 28 03:37:34.699161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-16530ce6-5758-4e0d-898c-524be1b9be88 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:34.699814 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-16530ce6-5758-4e0d-898c-524be1b9be88 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:34.707050 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16530ce6-5758-4e0d-898c-524be1b9be88 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:34.707700 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16530ce6-5758-4e0d-898c-524be1b9be88 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:34.708471 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16530ce6-5758-4e0d-898c-524be1b9be88 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:34.708908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16530ce6-5758-4e0d-898c-524be1b9be88 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:34.892227 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-16530ce6-5758-4e0d-898c-524be1b9be88 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:34.900425 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-16530ce6-5758-4e0d-898c-524be1b9be88 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.204979 May 28 03:37:34.901330 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 97/215] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:34 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:37:35.919266 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f401404a-02a0-4b0a-96cf-3ffa858b02b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:35.919985 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f401404a-02a0-4b0a-96cf-3ffa858b02b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:35.926163 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f401404a-02a0-4b0a-96cf-3ffa858b02b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:35.926603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f401404a-02a0-4b0a-96cf-3ffa858b02b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:35.927361 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f401404a-02a0-4b0a-96cf-3ffa858b02b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:35.927796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f401404a-02a0-4b0a-96cf-3ffa858b02b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:36.060639 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f401404a-02a0-4b0a-96cf-3ffa858b02b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:36.068900 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f401404a-02a0-4b0a-96cf-3ffa858b02b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.154150 May 28 03:37:36.078783 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 119/216] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:35 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:36.285403 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0b6ef24c-292e-4819-9074-53bb88d997b8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:37:36.285988 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0b6ef24c-292e-4819-9074-53bb88d997b8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:36.292309 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0b6ef24c-292e-4819-9074-53bb88d997b8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:36.292751 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0b6ef24c-292e-4819-9074-53bb88d997b8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:36.293538 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0b6ef24c-292e-4819-9074-53bb88d997b8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:36.293972 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0b6ef24c-292e-4819-9074-53bb88d997b8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:36.316882 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-0b6ef24c-292e-4819-9074-53bb88d997b8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lazy-loading 'flavor' on Instance uuid 35a9c645-590f-49e1-ab03-af4167f10547 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:37:36.850633 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0b6ef24c-292e-4819-9074-53bb88d997b8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "POST /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/action" status: 202 len: 0 microversion: 2.20 time: 0.568977 May 28 03:37:36.851248 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 98/217] 10.222.0.41 () {62 vars in 1318 bytes} [Sat May 28 03:37:36 2022] POST /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/action => generated 0 bytes in 570 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 28 03:37:36.859353 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6d63f970-67ba-49ae-bd36-e94f5670d55c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:36.860586 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-6d63f970-67ba-49ae-bd36-e94f5670d55c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:36.868821 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6d63f970-67ba-49ae-bd36-e94f5670d55c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:36.869308 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6d63f970-67ba-49ae-bd36-e94f5670d55c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:36.870148 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6d63f970-67ba-49ae-bd36-e94f5670d55c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:36.870743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6d63f970-67ba-49ae-bd36-e94f5670d55c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:36.999825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6d63f970-67ba-49ae-bd36-e94f5670d55c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:37.008166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6d63f970-67ba-49ae-bd36-e94f5670d55c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:37.009701 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6d63f970-67ba-49ae-bd36-e94f5670d55c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:37.010841 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6d63f970-67ba-49ae-bd36-e94f5670d55c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.155412 May 28 03:37:37.011396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 120/218] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:36 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:37.086539 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-11c36cb6-b835-412f-a9ef-04cb5672df29 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:37.087244 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-11c36cb6-b835-412f-a9ef-04cb5672df29 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:37.093704 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-11c36cb6-b835-412f-a9ef-04cb5672df29 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:37.094152 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-11c36cb6-b835-412f-a9ef-04cb5672df29 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:37.094918 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-11c36cb6-b835-412f-a9ef-04cb5672df29 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:37.095354 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-11c36cb6-b835-412f-a9ef-04cb5672df29 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:37.235598 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-11c36cb6-b835-412f-a9ef-04cb5672df29 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:37.243835 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-11c36cb6-b835-412f-a9ef-04cb5672df29 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.161828 May 28 03:37:37.244401 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 99/219] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:37 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:38.029230 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4454b8c1-844b-4b6e-9942-75e454345268 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:38.029837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4454b8c1-844b-4b6e-9942-75e454345268 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:38.036387 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4454b8c1-844b-4b6e-9942-75e454345268 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:38.036831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4454b8c1-844b-4b6e-9942-75e454345268 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:38.038892 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4454b8c1-844b-4b6e-9942-75e454345268 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:38.038892 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4454b8c1-844b-4b6e-9942-75e454345268 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:38.178459 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4454b8c1-844b-4b6e-9942-75e454345268 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:38.187901 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4454b8c1-844b-4b6e-9942-75e454345268 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:38.189468 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4454b8c1-844b-4b6e-9942-75e454345268 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:38.191132 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4454b8c1-844b-4b6e-9942-75e454345268 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.165367 May 28 03:37:38.191672 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 121/220] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:38 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:38.261501 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-41f9f326-6fb0-42d6-a004-f44b874057ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:38.262109 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-41f9f326-6fb0-42d6-a004-f44b874057ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:38.264407 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=30 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:38.270411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-41f9f326-6fb0-42d6-a004-f44b874057ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:38.270844 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-41f9f326-6fb0-42d6-a004-f44b874057ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:38.271598 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-41f9f326-6fb0-42d6-a004-f44b874057ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:38.272033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-41f9f326-6fb0-42d6-a004-f44b874057ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:38.408955 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-41f9f326-6fb0-42d6-a004-f44b874057ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:38.416924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-41f9f326-6fb0-42d6-a004-f44b874057ef tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.159294 May 28 03:37:38.417523 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 100/221] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:38 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:39.210847 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f725dbe2-7ade-48e6-b60d-617f23de3af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:39.212119 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f725dbe2-7ade-48e6-b60d-617f23de3af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:39.224159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f725dbe2-7ade-48e6-b60d-617f23de3af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:39.224702 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f725dbe2-7ade-48e6-b60d-617f23de3af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:39.225584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f725dbe2-7ade-48e6-b60d-617f23de3af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:39.235255 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f725dbe2-7ade-48e6-b60d-617f23de3af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:39.376242 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f725dbe2-7ade-48e6-b60d-617f23de3af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:39.384266 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f725dbe2-7ade-48e6-b60d-617f23de3af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:39.385843 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f725dbe2-7ade-48e6-b60d-617f23de3af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:39.387224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f725dbe2-7ade-48e6-b60d-617f23de3af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.181490 May 28 03:37:39.387834 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 122/222] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:39 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:39.433629 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-94134e06-d07c-4190-b7d9-ad455d0e5c44 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:39.434176 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-94134e06-d07c-4190-b7d9-ad455d0e5c44 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:39.440801 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-94134e06-d07c-4190-b7d9-ad455d0e5c44 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:39.441291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-94134e06-d07c-4190-b7d9-ad455d0e5c44 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:39.442053 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-94134e06-d07c-4190-b7d9-ad455d0e5c44 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:39.442497 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-94134e06-d07c-4190-b7d9-ad455d0e5c44 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:39.578071 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-94134e06-d07c-4190-b7d9-ad455d0e5c44 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:39.585949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-94134e06-d07c-4190-b7d9-ad455d0e5c44 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.156272 May 28 03:37:39.586548 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 101/223] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:39 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:40.405999 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-96b7a885-c2c4-4b68-b232-38dd078a286e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:40.406605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-96b7a885-c2c4-4b68-b232-38dd078a286e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:40.413360 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-96b7a885-c2c4-4b68-b232-38dd078a286e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:40.413802 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-96b7a885-c2c4-4b68-b232-38dd078a286e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:40.414566 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-96b7a885-c2c4-4b68-b232-38dd078a286e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:40.414995 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-96b7a885-c2c4-4b68-b232-38dd078a286e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:40.545687 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-96b7a885-c2c4-4b68-b232-38dd078a286e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:40.555775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-96b7a885-c2c4-4b68-b232-38dd078a286e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:40.555919 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-96b7a885-c2c4-4b68-b232-38dd078a286e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:40.557011 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-96b7a885-c2c4-4b68-b232-38dd078a286e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.155007 May 28 03:37:40.557661 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 123/224] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:40 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:40.603752 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3b7ddab3-e873-4776-9fb4-b99f7bca6996 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:40.604358 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3b7ddab3-e873-4776-9fb4-b99f7bca6996 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:40.610763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3b7ddab3-e873-4776-9fb4-b99f7bca6996 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:40.611220 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3b7ddab3-e873-4776-9fb4-b99f7bca6996 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:40.611980 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3b7ddab3-e873-4776-9fb4-b99f7bca6996 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:40.612433 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3b7ddab3-e873-4776-9fb4-b99f7bca6996 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:40.742990 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3b7ddab3-e873-4776-9fb4-b99f7bca6996 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:40.750642 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3b7ddab3-e873-4776-9fb4-b99f7bca6996 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.150769 May 28 03:37:40.751219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 102/225] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:40 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:41.574920 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-89cde561-4949-4ee9-8b19-8cfbd841a23b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:41.575521 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-89cde561-4949-4ee9-8b19-8cfbd841a23b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:41.577459 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=32 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:41.582650 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-89cde561-4949-4ee9-8b19-8cfbd841a23b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:41.583087 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-89cde561-4949-4ee9-8b19-8cfbd841a23b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:41.583843 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-89cde561-4949-4ee9-8b19-8cfbd841a23b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:41.584271 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-89cde561-4949-4ee9-8b19-8cfbd841a23b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:41.714290 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-89cde561-4949-4ee9-8b19-8cfbd841a23b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:41.717736 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:41.723117 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-89cde561-4949-4ee9-8b19-8cfbd841a23b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:41.724595 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-89cde561-4949-4ee9-8b19-8cfbd841a23b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:41.725762 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-89cde561-4949-4ee9-8b19-8cfbd841a23b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.154867 May 28 03:37:41.726319 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 124/226] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:41 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:41.767999 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-16f9b819-2467-4fc2-b7f3-8dc0ddfaf2c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:41.769095 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-16f9b819-2467-4fc2-b7f3-8dc0ddfaf2c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:41.775564 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16f9b819-2467-4fc2-b7f3-8dc0ddfaf2c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:41.776028 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16f9b819-2467-4fc2-b7f3-8dc0ddfaf2c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:41.776784 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16f9b819-2467-4fc2-b7f3-8dc0ddfaf2c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:41.777250 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16f9b819-2467-4fc2-b7f3-8dc0ddfaf2c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:41.907033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-16f9b819-2467-4fc2-b7f3-8dc0ddfaf2c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:41.914636 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-16f9b819-2467-4fc2-b7f3-8dc0ddfaf2c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.150413 May 28 03:37:41.915195 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 103/227] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:41 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:42.006643 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-41f1c411-15b1-4560-abf6-40ead8d9d33e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:42.007139 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-41f1c411-15b1-4560-abf6-40ead8d9d33e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:42.013437 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-41f1c411-15b1-4560-abf6-40ead8d9d33e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:42.013884 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-41f1c411-15b1-4560-abf6-40ead8d9d33e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:42.014646 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-41f1c411-15b1-4560-abf6-40ead8d9d33e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:42.015082 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-41f1c411-15b1-4560-abf6-40ead8d9d33e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:42.037986 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-41f1c411-15b1-4560-abf6-40ead8d9d33e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/ips" status: 200 len: 148 microversion: 2.1 time: 0.036053 May 28 03:37:42.038543 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 125/228] 10.222.0.41 () {58 vars in 1247 bytes} [Sat May 28 03:37:42 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/ips => generated 148 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:37:42.744773 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-77daafc2-b3f9-429d-8a6b-9bb23acc74e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:42.745407 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-77daafc2-b3f9-429d-8a6b-9bb23acc74e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:42.751593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-77daafc2-b3f9-429d-8a6b-9bb23acc74e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:42.752040 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-77daafc2-b3f9-429d-8a6b-9bb23acc74e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:42.752851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-77daafc2-b3f9-429d-8a6b-9bb23acc74e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:42.753323 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-77daafc2-b3f9-429d-8a6b-9bb23acc74e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:42.880817 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-77daafc2-b3f9-429d-8a6b-9bb23acc74e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:42.888659 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-77daafc2-b3f9-429d-8a6b-9bb23acc74e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:42.890198 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-77daafc2-b3f9-429d-8a6b-9bb23acc74e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:42.891354 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-77daafc2-b3f9-429d-8a6b-9bb23acc74e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.151027 May 28 03:37:42.891982 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 104/229] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:42 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:42.931440 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-af16e83e-0dea-4185-86df-4618e4d8bb76 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:42.932084 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-af16e83e-0dea-4185-86df-4618e4d8bb76 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:42.938792 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af16e83e-0dea-4185-86df-4618e4d8bb76 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:42.939231 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af16e83e-0dea-4185-86df-4618e4d8bb76 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:42.939989 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af16e83e-0dea-4185-86df-4618e4d8bb76 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:42.940423 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af16e83e-0dea-4185-86df-4618e4d8bb76 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:43.063557 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-af16e83e-0dea-4185-86df-4618e4d8bb76 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:43.071436 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-af16e83e-0dea-4185-86df-4618e4d8bb76 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.143809 May 28 03:37:43.072006 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 126/230] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:42 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:43.909437 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1569ac79-c415-4078-9e75-1efc6f804c94 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:43.910054 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1569ac79-c415-4078-9e75-1efc6f804c94 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:43.916562 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1569ac79-c415-4078-9e75-1efc6f804c94 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:43.917017 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1569ac79-c415-4078-9e75-1efc6f804c94 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:43.917833 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1569ac79-c415-4078-9e75-1efc6f804c94 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:43.918273 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1569ac79-c415-4078-9e75-1efc6f804c94 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:44.047944 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1569ac79-c415-4078-9e75-1efc6f804c94 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:44.056474 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1569ac79-c415-4078-9e75-1efc6f804c94 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:44.058006 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1569ac79-c415-4078-9e75-1efc6f804c94 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:44.059171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1569ac79-c415-4078-9e75-1efc6f804c94 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.153777 May 28 03:37:44.059786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 105/231] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:43 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:44.088180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c81e7943-9fe8-4859-b918-bc2915614728 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:44.088904 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c81e7943-9fe8-4859-b918-bc2915614728 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:44.094927 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c81e7943-9fe8-4859-b918-bc2915614728 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:44.095376 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c81e7943-9fe8-4859-b918-bc2915614728 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:44.096137 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c81e7943-9fe8-4859-b918-bc2915614728 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:44.096572 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c81e7943-9fe8-4859-b918-bc2915614728 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:44.226716 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c81e7943-9fe8-4859-b918-bc2915614728 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:44.234215 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c81e7943-9fe8-4859-b918-bc2915614728 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.149895 May 28 03:37:44.234785 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 127/232] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:44 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:44.468080 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4114648a-27c3-4631-a965-68bdc3c116f9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:44.469990 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4114648a-27c3-4631-a965-68bdc3c116f9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:44.474496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4114648a-27c3-4631-a965-68bdc3c116f9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:44.474945 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4114648a-27c3-4631-a965-68bdc3c116f9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:44.475701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4114648a-27c3-4631-a965-68bdc3c116f9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:44.476135 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4114648a-27c3-4631-a965-68bdc3c116f9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:44.499329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4114648a-27c3-4631-a965-68bdc3c116f9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/ips" status: 200 len: 145 microversion: 2.1 time: 0.034980 May 28 03:37:44.499898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 106/233] 10.222.0.41 () {58 vars in 1247 bytes} [Sat May 28 03:37:44 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/ips => generated 145 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:37:44.741442 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-cd225e11-0c82-4e53-aea5-c3ad077bff0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cdb718c8-6afe-412f-b14b-ddce4cf83a7a"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:37:44.743768 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-cd225e11-0c82-4e53-aea5-c3ad077bff0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:44.750821 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cd225e11-0c82-4e53-aea5-c3ad077bff0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:44.751759 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cd225e11-0c82-4e53-aea5-c3ad077bff0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:44.752741 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cd225e11-0c82-4e53-aea5-c3ad077bff0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:44.754022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cd225e11-0c82-4e53-aea5-c3ad077bff0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:45.077449 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4eb6c62d-438d-4224-a93a-b107ada84af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:45.078519 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4eb6c62d-438d-4224-a93a-b107ada84af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:45.085676 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4eb6c62d-438d-4224-a93a-b107ada84af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:45.086428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4eb6c62d-438d-4224-a93a-b107ada84af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:45.087251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4eb6c62d-438d-4224-a93a-b107ada84af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:45.087682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4eb6c62d-438d-4224-a93a-b107ada84af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:45.223796 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4eb6c62d-438d-4224-a93a-b107ada84af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:45.232857 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4eb6c62d-438d-4224-a93a-b107ada84af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:45.234374 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4eb6c62d-438d-4224-a93a-b107ada84af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:45.235519 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4eb6c62d-438d-4224-a93a-b107ada84af4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.161966 May 28 03:37:45.238213 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 107/234] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:45 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:45.253887 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a4b1d34a-b6dc-4ea6-b2ff-1fa65501d426 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:45.256219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a4b1d34a-b6dc-4ea6-b2ff-1fa65501d426 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:45.260975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a4b1d34a-b6dc-4ea6-b2ff-1fa65501d426 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:45.261564 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a4b1d34a-b6dc-4ea6-b2ff-1fa65501d426 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:45.262392 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a4b1d34a-b6dc-4ea6-b2ff-1fa65501d426 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:45.262923 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a4b1d34a-b6dc-4ea6-b2ff-1fa65501d426 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:45.422491 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a4b1d34a-b6dc-4ea6-b2ff-1fa65501d426 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:45.427234 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a4b1d34a-b6dc-4ea6-b2ff-1fa65501d426 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.176961 May 28 03:37:45.427802 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 108/235] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:45 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:45.756365 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-cd225e11-0c82-4e53-aea5-c3ad077bff0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.029574 May 28 03:37:45.757327 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 128/236] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:37:44 2022] POST /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments => generated 194 bytes in 1031 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:37:46.254286 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2bf87782-c75b-4b0c-83a6-24981a4997ae tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:46.255033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2bf87782-c75b-4b0c-83a6-24981a4997ae tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:46.261518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2bf87782-c75b-4b0c-83a6-24981a4997ae tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:46.261954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2bf87782-c75b-4b0c-83a6-24981a4997ae tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:46.266295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2bf87782-c75b-4b0c-83a6-24981a4997ae tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:46.266295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2bf87782-c75b-4b0c-83a6-24981a4997ae tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:46.278394 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:46.400183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2bf87782-c75b-4b0c-83a6-24981a4997ae tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:46.408481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2bf87782-c75b-4b0c-83a6-24981a4997ae tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:46.410191 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2bf87782-c75b-4b0c-83a6-24981a4997ae tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:46.411394 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2bf87782-c75b-4b0c-83a6-24981a4997ae tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.160910 May 28 03:37:46.411947 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 109/237] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:46 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:46.446813 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-09c22c36-0f28-4647-afc8-e8009dc69753 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:46.447728 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-09c22c36-0f28-4647-afc8-e8009dc69753 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:46.453925 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09c22c36-0f28-4647-afc8-e8009dc69753 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:46.454374 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09c22c36-0f28-4647-afc8-e8009dc69753 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:46.455149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09c22c36-0f28-4647-afc8-e8009dc69753 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:46.455567 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09c22c36-0f28-4647-afc8-e8009dc69753 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:46.613079 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-09c22c36-0f28-4647-afc8-e8009dc69753 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:46.624357 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-09c22c36-0f28-4647-afc8-e8009dc69753 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.182822 May 28 03:37:46.624948 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 129/238] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:46 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:47.430003 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1dfbc676-6f27-4735-b3cc-0cf0e3b30b1e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:47.431314 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1dfbc676-6f27-4735-b3cc-0cf0e3b30b1e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:47.438564 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dfbc676-6f27-4735-b3cc-0cf0e3b30b1e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:47.439091 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dfbc676-6f27-4735-b3cc-0cf0e3b30b1e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:47.440066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dfbc676-6f27-4735-b3cc-0cf0e3b30b1e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:47.440497 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dfbc676-6f27-4735-b3cc-0cf0e3b30b1e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:47.593725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1dfbc676-6f27-4735-b3cc-0cf0e3b30b1e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:47.604417 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1dfbc676-6f27-4735-b3cc-0cf0e3b30b1e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:47.606377 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1dfbc676-6f27-4735-b3cc-0cf0e3b30b1e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:47.607716 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1dfbc676-6f27-4735-b3cc-0cf0e3b30b1e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.182376 May 28 03:37:47.608526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 110/239] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:47 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:37:47.642695 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-60ec7c50-a6c5-49db-bbd0-60348bea1b5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:47.643857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-60ec7c50-a6c5-49db-bbd0-60348bea1b5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:47.655179 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-60ec7c50-a6c5-49db-bbd0-60348bea1b5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:47.657712 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-60ec7c50-a6c5-49db-bbd0-60348bea1b5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:47.658411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-60ec7c50-a6c5-49db-bbd0-60348bea1b5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:47.658845 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-60ec7c50-a6c5-49db-bbd0-60348bea1b5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:47.846834 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-60ec7c50-a6c5-49db-bbd0-60348bea1b5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:47.854804 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-60ec7c50-a6c5-49db-bbd0-60348bea1b5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.216789 May 28 03:37:47.855367 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 130/240] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:47 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:48.626763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1393b0f3-f1e5-4752-8baa-e1bbbbbb6190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:48.633258 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1393b0f3-f1e5-4752-8baa-e1bbbbbb6190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:48.646661 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1393b0f3-f1e5-4752-8baa-e1bbbbbb6190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:48.651630 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1393b0f3-f1e5-4752-8baa-e1bbbbbb6190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:48.652721 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1393b0f3-f1e5-4752-8baa-e1bbbbbb6190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:48.653467 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1393b0f3-f1e5-4752-8baa-e1bbbbbb6190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:48.784731 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-7ca94d88-f1cf-421c-a3c4-19a1f730ce49 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "181f3f5a-3875-45bf-a1fb-7c09612282f4"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:37:48.788894 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-7ca94d88-f1cf-421c-a3c4-19a1f730ce49 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:48.797744 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7ca94d88-f1cf-421c-a3c4-19a1f730ce49 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:48.798226 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7ca94d88-f1cf-421c-a3c4-19a1f730ce49 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:48.799069 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7ca94d88-f1cf-421c-a3c4-19a1f730ce49 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:48.799677 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7ca94d88-f1cf-421c-a3c4-19a1f730ce49 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:48.858047 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1393b0f3-f1e5-4752-8baa-e1bbbbbb6190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:48.867845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1393b0f3-f1e5-4752-8baa-e1bbbbbb6190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:48.870141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1393b0f3-f1e5-4752-8baa-e1bbbbbb6190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:48.875476 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1393b0f3-f1e5-4752-8baa-e1bbbbbb6190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.249077 May 28 03:37:48.882397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 111/241] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:48 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:48.891924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c3b1a18c-13ea-49ee-8352-f66c80851796 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:48.892686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c3b1a18c-13ea-49ee-8352-f66c80851796 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:48.899414 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c3b1a18c-13ea-49ee-8352-f66c80851796 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:48.900181 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c3b1a18c-13ea-49ee-8352-f66c80851796 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:48.901206 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c3b1a18c-13ea-49ee-8352-f66c80851796 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:48.901846 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c3b1a18c-13ea-49ee-8352-f66c80851796 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:49.189094 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c3b1a18c-13ea-49ee-8352-f66c80851796 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:49.201673 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c3b1a18c-13ea-49ee-8352-f66c80851796 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.322922 May 28 03:37:49.202183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 112/242] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:48 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:49.646558 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-7ca94d88-f1cf-421c-a3c4-19a1f730ce49 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "POST /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.864529 May 28 03:37:49.650291 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 131/243] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:37:48 2022] POST /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/os-volume_attachments => generated 194 bytes in 868 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:37:49.900808 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6b458ccd-2949-4d81-b0b0-0dcb9107d020 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:49.902074 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6b458ccd-2949-4d81-b0b0-0dcb9107d020 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:49.907971 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6b458ccd-2949-4d81-b0b0-0dcb9107d020 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:49.908417 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6b458ccd-2949-4d81-b0b0-0dcb9107d020 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:49.909255 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6b458ccd-2949-4d81-b0b0-0dcb9107d020 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:49.909686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6b458ccd-2949-4d81-b0b0-0dcb9107d020 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:50.054796 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6b458ccd-2949-4d81-b0b0-0dcb9107d020 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:50.062577 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6b458ccd-2949-4d81-b0b0-0dcb9107d020 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:50.064066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6b458ccd-2949-4d81-b0b0-0dcb9107d020 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:50.065237 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6b458ccd-2949-4d81-b0b0-0dcb9107d020 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.169339 May 28 03:37:50.065792 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 113/244] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:49 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:50.221017 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9ec66527-faa7-4953-9ad9-905b52d1487a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:50.221825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9ec66527-faa7-4953-9ad9-905b52d1487a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:50.231035 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9ec66527-faa7-4953-9ad9-905b52d1487a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:50.231507 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9ec66527-faa7-4953-9ad9-905b52d1487a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:50.232299 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9ec66527-faa7-4953-9ad9-905b52d1487a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:50.232748 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9ec66527-faa7-4953-9ad9-905b52d1487a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:50.377257 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9ec66527-faa7-4953-9ad9-905b52d1487a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:50.384900 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9ec66527-faa7-4953-9ad9-905b52d1487a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.168807 May 28 03:37:50.385563 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 132/245] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:50 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:51.084947 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1dae5d78-5573-4b03-8137-5926391b6db2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:51.085866 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1dae5d78-5573-4b03-8137-5926391b6db2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:51.092690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dae5d78-5573-4b03-8137-5926391b6db2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:51.093460 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dae5d78-5573-4b03-8137-5926391b6db2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:51.094307 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dae5d78-5573-4b03-8137-5926391b6db2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:51.094751 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dae5d78-5573-4b03-8137-5926391b6db2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:51.261056 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1dae5d78-5573-4b03-8137-5926391b6db2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:51.275872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1dae5d78-5573-4b03-8137-5926391b6db2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:51.278006 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1dae5d78-5573-4b03-8137-5926391b6db2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:51.279176 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1dae5d78-5573-4b03-8137-5926391b6db2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.199730 May 28 03:37:51.279939 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 114/246] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:51 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:51.402267 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e44d4b8f-1efa-4397-ac0a-1a68003e8bf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:51.402971 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e44d4b8f-1efa-4397-ac0a-1a68003e8bf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:51.409804 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e44d4b8f-1efa-4397-ac0a-1a68003e8bf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:51.410501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e44d4b8f-1efa-4397-ac0a-1a68003e8bf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:51.411277 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e44d4b8f-1efa-4397-ac0a-1a68003e8bf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:51.411725 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e44d4b8f-1efa-4397-ac0a-1a68003e8bf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:51.569621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e44d4b8f-1efa-4397-ac0a-1a68003e8bf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:51.577803 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e44d4b8f-1efa-4397-ac0a-1a68003e8bf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.179508 May 28 03:37:51.579344 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 133/247] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:51 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:52.298025 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-eabd4e18-4295-4e7b-a71c-3dd287ad86b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:52.299166 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-eabd4e18-4295-4e7b-a71c-3dd287ad86b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:52.306792 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eabd4e18-4295-4e7b-a71c-3dd287ad86b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:52.307231 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eabd4e18-4295-4e7b-a71c-3dd287ad86b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:52.308011 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eabd4e18-4295-4e7b-a71c-3dd287ad86b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:52.308453 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eabd4e18-4295-4e7b-a71c-3dd287ad86b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:52.475012 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-eabd4e18-4295-4e7b-a71c-3dd287ad86b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:52.482734 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-eabd4e18-4295-4e7b-a71c-3dd287ad86b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:52.484249 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-eabd4e18-4295-4e7b-a71c-3dd287ad86b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:52.485456 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-eabd4e18-4295-4e7b-a71c-3dd287ad86b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.191382 May 28 03:37:52.486016 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 115/248] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:52 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:37:52.595680 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2608534b-7463-4e9c-87c8-8825a4a0e7aa tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:52.596386 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2608534b-7463-4e9c-87c8-8825a4a0e7aa tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:52.603314 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2608534b-7463-4e9c-87c8-8825a4a0e7aa tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:52.603760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2608534b-7463-4e9c-87c8-8825a4a0e7aa tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:52.604523 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2608534b-7463-4e9c-87c8-8825a4a0e7aa tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:52.604953 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2608534b-7463-4e9c-87c8-8825a4a0e7aa tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:52.751714 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-2608534b-7463-4e9c-87c8-8825a4a0e7aa tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:52.759063 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2608534b-7463-4e9c-87c8-8825a4a0e7aa tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.167279 May 28 03:37:52.759632 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 134/249] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:52 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:53.506610 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-05c2ef6a-1b89-49d7-8e06-ef51b2a7a57e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:53.508244 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-05c2ef6a-1b89-49d7-8e06-ef51b2a7a57e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:53.513605 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-05c2ef6a-1b89-49d7-8e06-ef51b2a7a57e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:53.514049 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-05c2ef6a-1b89-49d7-8e06-ef51b2a7a57e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:53.514812 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-05c2ef6a-1b89-49d7-8e06-ef51b2a7a57e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:53.515407 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-05c2ef6a-1b89-49d7-8e06-ef51b2a7a57e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:53.580935 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-592697900", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "networks": [{"uuid": "d0e4e4f5-23db-4c8c-a345-0058f3ae5e11"}]}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:37:53.647993 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] validate_networks() for [('d0e4e4f5-23db-4c8c-a345-0058f3ae5e11', None, None, None, None, None)] {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:37:53.674205 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-05c2ef6a-1b89-49d7-8e06-ef51b2a7a57e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:53.703903 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-05c2ef6a-1b89-49d7-8e06-ef51b2a7a57e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:53.705437 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-05c2ef6a-1b89-49d7-8e06-ef51b2a7a57e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:53.710469 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-05c2ef6a-1b89-49d7-8e06-ef51b2a7a57e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1746 microversion: 2.20 time: 0.210492 May 28 03:37:53.713022 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 116/250] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:53 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1746 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:53.777407 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a59ca158-5dd9-4c16-be2d-9b99ecc312c2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:53.777970 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a59ca158-5dd9-4c16-be2d-9b99ecc312c2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:53.791202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a59ca158-5dd9-4c16-be2d-9b99ecc312c2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:53.791662 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a59ca158-5dd9-4c16-be2d-9b99ecc312c2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:53.792519 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a59ca158-5dd9-4c16-be2d-9b99ecc312c2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:53.792954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a59ca158-5dd9-4c16-be2d-9b99ecc312c2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:53.964356 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a59ca158-5dd9-4c16-be2d-9b99ecc312c2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:53.980938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a59ca158-5dd9-4c16-be2d-9b99ecc312c2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.208454 May 28 03:37:53.984330 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 117/251] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:53 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:53.986886 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d1440361-87ef-43d8-8c67-2f400d0cf016 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: .version_select at 0x7feb1feb0550>, body: {"keypair": {"name": "tempest-keypair-1533452588", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFZg+SGuMbYM1hvAoZywoENxTKGmL7rAf6Iib7UrsySS0g1U9gVSGeSl5XLdCTDxx48v2SZ9EuNV40HfUonD/rQwTte8KucRtOQTlN5rWBLSymUUyZdpGNfiG3IzU6/hoQ=="}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:37:54.003512 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-d1440361-87ef-43d8-8c67-2f400d0cf016 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:37:54.007459 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-d1440361-87ef-43d8-8c67-2f400d0cf016 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for user 9d420e4b554e4deab21b89094e0547ca and project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:37:54.067254 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:37:54.067679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:37:54.068144 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:37:54.068550 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:37:54.068955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:37:54.078067 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:54.078548 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:54.079344 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:54.079775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:54.102555 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Getting quotas for project ccf50ff938ce4fdb8d24871054e0015d. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:37:54.106838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Getting quotas for user db802295be064253ae59efcbfd12be8f and project ccf50ff938ce4fdb8d24871054e0015d. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:37:54.118614 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:37:54.120841 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:37:54.125637 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d1440361-87ef-43d8-8c67-2f400d0cf016 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.142680 May 28 03:37:54.126246 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 118/252] 10.222.0.41 () {60 vars in 1158 bytes} [Sat May 28 03:37:53 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:37:54.322343 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-dfbe9608-e047-468f-89e4-805e81ff35ab tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.738212 May 28 03:37:54.322343 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 135/253] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:37:53 2022] POST /compute/v2.1/servers => generated 380 bytes in 741 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 28 03:37:54.328397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7d532a1a-19e2-43a3-841c-04957ccef66f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:54.329117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7d532a1a-19e2-43a3-841c-04957ccef66f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:54.416215 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7d532a1a-19e2-43a3-841c-04957ccef66f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:54.425293 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7d532a1a-19e2-43a3-841c-04957ccef66f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1213 microversion: 2.1 time: 0.101509 May 28 03:37:54.430428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 119/254] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:54 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1213 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:54.735386 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-17c15a3a-73b7-43b4-8ad1-e73b9b13f791 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:54.736022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-17c15a3a-73b7-43b4-8ad1-e73b9b13f791 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:54.749052 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17c15a3a-73b7-43b4-8ad1-e73b9b13f791 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:54.749714 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17c15a3a-73b7-43b4-8ad1-e73b9b13f791 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:54.750676 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17c15a3a-73b7-43b4-8ad1-e73b9b13f791 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:54.751244 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17c15a3a-73b7-43b4-8ad1-e73b9b13f791 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:54.921754 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-17c15a3a-73b7-43b4-8ad1-e73b9b13f791 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:54.927166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-17c15a3a-73b7-43b4-8ad1-e73b9b13f791 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:54.928743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-17c15a3a-73b7-43b4-8ad1-e73b9b13f791 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:54.929931 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-17c15a3a-73b7-43b4-8ad1-e73b9b13f791 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1746 microversion: 2.20 time: 0.202795 May 28 03:37:54.930487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 136/255] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:54 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1746 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:54.999784 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6f93fb01-51c4-43a0-a93f-b0e947e1774f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:55.000624 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6f93fb01-51c4-43a0-a93f-b0e947e1774f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:55.009678 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f93fb01-51c4-43a0-a93f-b0e947e1774f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:55.010160 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f93fb01-51c4-43a0-a93f-b0e947e1774f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:55.011116 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f93fb01-51c4-43a0-a93f-b0e947e1774f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:55.011899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f93fb01-51c4-43a0-a93f-b0e947e1774f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:55.161979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6f93fb01-51c4-43a0-a93f-b0e947e1774f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:55.181430 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6f93fb01-51c4-43a0-a93f-b0e947e1774f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.178829 May 28 03:37:55.181430 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 120/256] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:54 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:55.445675 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d970e714-9614-4230-9dba-f40cf7aac7af tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:55.448129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d970e714-9614-4230-9dba-f40cf7aac7af tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:55.505608 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d970e714-9614-4230-9dba-f40cf7aac7af tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:55.513901 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d970e714-9614-4230-9dba-f40cf7aac7af tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1213 microversion: 2.1 time: 0.073840 May 28 03:37:55.514508 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 137/257] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:55 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1213 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:55.948483 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c4b35f13-f9c5-4544-9a4d-f76b9fa96778 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:55.949226 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c4b35f13-f9c5-4544-9a4d-f76b9fa96778 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:55.955227 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c4b35f13-f9c5-4544-9a4d-f76b9fa96778 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:55.955674 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c4b35f13-f9c5-4544-9a4d-f76b9fa96778 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:55.956621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c4b35f13-f9c5-4544-9a4d-f76b9fa96778 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:55.957191 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c4b35f13-f9c5-4544-9a4d-f76b9fa96778 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:56.174214 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c4b35f13-f9c5-4544-9a4d-f76b9fa96778 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:56.189472 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c4b35f13-f9c5-4544-9a4d-f76b9fa96778 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:56.192333 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c4b35f13-f9c5-4544-9a4d-f76b9fa96778 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:56.197663 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1cbf7013-0db8-4981-a4e5-b4bf85874197 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:56.198148 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c4b35f13-f9c5-4544-9a4d-f76b9fa96778 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1746 microversion: 2.20 time: 0.249255 May 28 03:37:56.198273 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1cbf7013-0db8-4981-a4e5-b4bf85874197 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:56.198365 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 121/258] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:55 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1746 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:56.211288 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1cbf7013-0db8-4981-a4e5-b4bf85874197 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:56.211863 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1cbf7013-0db8-4981-a4e5-b4bf85874197 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:56.212753 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1cbf7013-0db8-4981-a4e5-b4bf85874197 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:56.213780 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1cbf7013-0db8-4981-a4e5-b4bf85874197 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:56.355638 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1cbf7013-0db8-4981-a4e5-b4bf85874197 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:56.370934 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1cbf7013-0db8-4981-a4e5-b4bf85874197 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.174629 May 28 03:37:56.370934 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 138/259] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:56 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:37:56.536997 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d0740830-f5bf-4270-acf3-0a892ce8a288 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:56.537933 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d0740830-f5bf-4270-acf3-0a892ce8a288 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:56.547874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d0740830-f5bf-4270-acf3-0a892ce8a288 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:56.548328 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d0740830-f5bf-4270-acf3-0a892ce8a288 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:56.549129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d0740830-f5bf-4270-acf3-0a892ce8a288 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:56.549586 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d0740830-f5bf-4270-acf3-0a892ce8a288 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:56.632872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d0740830-f5bf-4270-acf3-0a892ce8a288 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:56.640481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d0740830-f5bf-4270-acf3-0a892ce8a288 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1209 microversion: 2.1 time: 0.112447 May 28 03:37:56.641056 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 122/260] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:56 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1209 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:56.953634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-92355119", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ee7025a6-8632-48bc-9a4d-e0b6d8e98a79"}], "security_groups": [{"name": "tempest-securitygroup--1105998138"}], "key_name": "tempest-keypair-1533452588", "user_data": "}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:37:57.052774 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] validate_networks() for [('ee7025a6-8632-48bc-9a4d-e0b6d8e98a79', None, None, None, None, None)] {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:37:57.201838 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-cc6077d0-6e45-427d-a450-8c095e0153d2 req-de311165-6d38-4640-83e8-4b06e1592184 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35a9c645-590f-49e1-ab03-af4167f10547", "name": "network-vif-plugged", "status": "completed", "tag": "97df7472-c1e5-4bbd-80b5-6efb1965aeb9"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:37:57.219787 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-cc6077d0-6e45-427d-a450-8c095e0153d2 req-de311165-6d38-4640-83e8-4b06e1592184 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:57.227187 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-cc6077d0-6e45-427d-a450-8c095e0153d2 req-de311165-6d38-4640-83e8-4b06e1592184 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:57.268641 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-cc6077d0-6e45-427d-a450-8c095e0153d2 req-de311165-6d38-4640-83e8-4b06e1592184 service nova] Creating event network-vif-plugged:97df7472-c1e5-4bbd-80b5-6efb1965aeb9 for instance 35a9c645-590f-49e1-ab03-af4167f10547 on hv-cinder-84336 May 28 03:37:57.283395 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-cc6077d0-6e45-427d-a450-8c095e0153d2 req-de311165-6d38-4640-83e8-4b06e1592184 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087044 May 28 03:37:57.289970 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 123/261] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:37:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:37:57.296759 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d54f74ee-bba9-4e7c-a89a-6859f0be2c44 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:57.297792 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d54f74ee-bba9-4e7c-a89a-6859f0be2c44 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:57.309342 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d54f74ee-bba9-4e7c-a89a-6859f0be2c44 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:57.310537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d54f74ee-bba9-4e7c-a89a-6859f0be2c44 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:57.311565 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d54f74ee-bba9-4e7c-a89a-6859f0be2c44 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:57.312356 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d54f74ee-bba9-4e7c-a89a-6859f0be2c44 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:57.595168 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:37:57.595622 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:37:57.596033 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:37:57.596595 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:37:57.597183 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:37:57.619672 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:57.620116 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:57.620907 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:57.621359 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:57.637033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d54f74ee-bba9-4e7c-a89a-6859f0be2c44 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:57.649438 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d54f74ee-bba9-4e7c-a89a-6859f0be2c44 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:57.652587 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d54f74ee-bba9-4e7c-a89a-6859f0be2c44 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:57.654775 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d54f74ee-bba9-4e7c-a89a-6859f0be2c44 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1746 microversion: 2.20 time: 0.365798 May 28 03:37:57.655197 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:37:57.655955 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 124/262] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:57 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1746 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:57.661932 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for user 9d420e4b554e4deab21b89094e0547ca and project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:37:57.672398 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d9a49ede-2fe8-40f0-9f38-5241ddb5125f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:57.673292 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d9a49ede-2fe8-40f0-9f38-5241ddb5125f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:57.674024 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:37:57.676435 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:37:57.680522 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d9a49ede-2fe8-40f0-9f38-5241ddb5125f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:57.681266 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d9a49ede-2fe8-40f0-9f38-5241ddb5125f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:57.682256 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d9a49ede-2fe8-40f0-9f38-5241ddb5125f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:57.682875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d9a49ede-2fe8-40f0-9f38-5241ddb5125f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:57.837019 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-70564f24-0339-4d4a-8439-ab678835c0fe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.887554 May 28 03:37:57.839193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 139/263] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:37:56 2022] POST /compute/v2.1/servers => generated 402 bytes in 890 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:37:57.849940 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-14e43ac8-edfa-429e-87f8-1896189f8eb4 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:57.849940 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-14e43ac8-edfa-429e-87f8-1896189f8eb4 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:57.860410 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-14e43ac8-edfa-429e-87f8-1896189f8eb4 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:57.861365 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-14e43ac8-edfa-429e-87f8-1896189f8eb4 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:57.862513 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-14e43ac8-edfa-429e-87f8-1896189f8eb4 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:57.863028 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-14e43ac8-edfa-429e-87f8-1896189f8eb4 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:57.922232 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d9a49ede-2fe8-40f0-9f38-5241ddb5125f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:57.934350 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d9a49ede-2fe8-40f0-9f38-5241ddb5125f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1502 microversion: 2.1 time: 0.276670 May 28 03:37:57.938392 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 125/264] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:57 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1502 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:57.958198 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c618fc64-8d02-40b2-912c-bb14191df42d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:57.959303 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c618fc64-8d02-40b2-912c-bb14191df42d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:57.991349 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-14e43ac8-edfa-429e-87f8-1896189f8eb4 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:58.002676 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-14e43ac8-edfa-429e-87f8-1896189f8eb4 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1271 microversion: 2.1 time: 0.161195 May 28 03:37:58.003119 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 140/265] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:57 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1271 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:58.016079 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-743ec5f9-9e3e-4e29-9ee6-49ad959e6428 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "eab5374a-d430-4e48-bf4a-0dee53d9b4a0"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:37:58.017235 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-743ec5f9-9e3e-4e29-9ee6-49ad959e6428 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:58.029703 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-743ec5f9-9e3e-4e29-9ee6-49ad959e6428 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:58.031398 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-743ec5f9-9e3e-4e29-9ee6-49ad959e6428 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:58.034865 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-743ec5f9-9e3e-4e29-9ee6-49ad959e6428 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:58.036346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-743ec5f9-9e3e-4e29-9ee6-49ad959e6428 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:58.094720 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:58.112390 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-743ec5f9-9e3e-4e29-9ee6-49ad959e6428 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] HTTP exception thrown: Cannot 'attach_volume' instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 while it is in vm_state rescued May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-743ec5f9-9e3e-4e29-9ee6-49ad959e6428 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Request method failure captured: May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: POST /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8/os-volume_attachments HTTP/1.1 May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 74 May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZioFSMjchJLpIZqod5q-7lgl8fEY36gE52nylmpmtlBCMGq9sB1-y2xrNX1ntFqjnFW5i92fC2liYIvh0AHl9bCuVQTnAqKQCbialBgg6tn7xuEEa8ZKcsvKvenrdvzcDN3coDLFL7-jztA0Tb3mMeshJqy19Ht6r4OU5cn2edXS_E May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 5cab8255c6134aebb9c93493a2342cb1 May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:37:58.118085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 5cab8255c6134aebb9c93493a2342cb1 May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-ServerRescueNegativeTestJSON-968064324-project May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: b6709eed72ca46468fd298929ae0f764 May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-ServerRescueNegativeTestJSON-968064324-project May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {"volumeAttachment": {"volumeId": "eab5374a-d430-4e48-bf4a-0dee53d9b4a0"}} May 28 03:37:58.119443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Cannot 'attach_volume' instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 while it is in vm_state rescued May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'server_id': '3b4a0f56-eb59-4f13-a90a-442b9a2e0af8', 'body': {'volumeAttachment': {'volumeId': 'eab5374a-d430-4e48-bf4a-0dee53d9b4a0'}}} May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 in vm_state rescued. Cannot attach_volume while the instance is in this state. May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:37:58.120167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 while it is in vm_state rescued May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:37:58.125322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:37:58.128531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:37:58.128531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:37:58.128531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 while it is in vm_state rescued May 28 03:37:58.128531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:37:58.128531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-743ec5f9-9e3e-4e29-9ee6-49ad959e6428 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Returning 409 to user: Cannot 'attach_volume' instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 while it is in vm_state rescued {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:37:58.128531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-743ec5f9-9e3e-4e29-9ee6-49ad959e6428 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "POST /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.115576 May 28 03:37:58.128531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 141/266] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:37:58 2022] POST /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8/os-volume_attachments => generated 152 bytes in 121 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 28 03:37:58.137630 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3c284cce-bfc0-4217-9b09-68d4615e7268 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:37:58.138164 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3c284cce-bfc0-4217-9b09-68d4615e7268 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:58.170899 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c284cce-bfc0-4217-9b09-68d4615e7268 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:58.175161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c284cce-bfc0-4217-9b09-68d4615e7268 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:58.175161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c284cce-bfc0-4217-9b09-68d4615e7268 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:58.175161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c284cce-bfc0-4217-9b09-68d4615e7268 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:58.180121 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c618fc64-8d02-40b2-912c-bb14191df42d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:58.188706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c618fc64-8d02-40b2-912c-bb14191df42d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1240 microversion: 2.1 time: 0.247858 May 28 03:37:58.189867 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 126/267] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:57 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1240 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:58.535351 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3c284cce-bfc0-4217-9b09-68d4615e7268 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "POST /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8/action" status: 202 len: 0 microversion: 2.1 time: 0.408315 May 28 03:37:58.538938 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 142/268] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:37:58 2022] POST /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8/action => generated 0 bytes in 410 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:37:58.550834 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-edd42df3-f089-497c-bd61-eb158057b50d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:58.551891 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-edd42df3-f089-497c-bd61-eb158057b50d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:58.566745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-edd42df3-f089-497c-bd61-eb158057b50d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:58.567364 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-edd42df3-f089-497c-bd61-eb158057b50d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:58.569505 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-edd42df3-f089-497c-bd61-eb158057b50d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:58.570090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-edd42df3-f089-497c-bd61-eb158057b50d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:58.677795 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ec4c1650-a0e2-4002-b2f0-34742fe87381 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:58.679366 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ec4c1650-a0e2-4002-b2f0-34742fe87381 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:58.694826 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ec4c1650-a0e2-4002-b2f0-34742fe87381 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:58.695278 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ec4c1650-a0e2-4002-b2f0-34742fe87381 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:58.696067 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ec4c1650-a0e2-4002-b2f0-34742fe87381 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:58.696503 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ec4c1650-a0e2-4002-b2f0-34742fe87381 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:58.903754 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-edd42df3-f089-497c-bd61-eb158057b50d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:58.913502 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-edd42df3-f089-497c-bd61-eb158057b50d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1510 microversion: 2.1 time: 0.371063 May 28 03:37:58.915180 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 127/269] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:58 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1510 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:59.021894 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-baae3a86-e43a-4bdc-b128-4c6d17e1cc2a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:59.022879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-baae3a86-e43a-4bdc-b128-4c6d17e1cc2a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:59.032157 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-baae3a86-e43a-4bdc-b128-4c6d17e1cc2a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:59.032959 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-baae3a86-e43a-4bdc-b128-4c6d17e1cc2a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:59.034009 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-baae3a86-e43a-4bdc-b128-4c6d17e1cc2a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:59.034955 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-baae3a86-e43a-4bdc-b128-4c6d17e1cc2a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:59.300695 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ec4c1650-a0e2-4002-b2f0-34742fe87381 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:59.326191 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ec4c1650-a0e2-4002-b2f0-34742fe87381 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:59.327791 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ec4c1650-a0e2-4002-b2f0-34742fe87381 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:59.328960 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ec4c1650-a0e2-4002-b2f0-34742fe87381 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.657816 May 28 03:37:59.329867 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 143/270] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:37:58 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 660 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:37:59.334887 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-23e60ac1-f1c1-4636-aa8f-127de89c5690 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:59.335586 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-23e60ac1-f1c1-4636-aa8f-127de89c5690 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:59.358153 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-23e60ac1-f1c1-4636-aa8f-127de89c5690 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:59.358611 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-23e60ac1-f1c1-4636-aa8f-127de89c5690 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:59.359389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-23e60ac1-f1c1-4636-aa8f-127de89c5690 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:59.359874 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-23e60ac1-f1c1-4636-aa8f-127de89c5690 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:59.393995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-baae3a86-e43a-4bdc-b128-4c6d17e1cc2a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:59.406696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-baae3a86-e43a-4bdc-b128-4c6d17e1cc2a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.391014 May 28 03:37:59.407302 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 128/271] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:59 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:59.483108 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-23e60ac1-f1c1-4636-aa8f-127de89c5690 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:37:59.493695 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-23e60ac1-f1c1-4636-aa8f-127de89c5690 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1244 microversion: 2.1 time: 0.162768 May 28 03:37:59.498647 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 144/272] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:59 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1244 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:37:59.935825 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8d0a2ac7-9f8d-4be0-ae58-bbc7d86eec5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:37:59.938595 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8d0a2ac7-9f8d-4be0-ae58-bbc7d86eec5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:37:59.965721 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8d0a2ac7-9f8d-4be0-ae58-bbc7d86eec5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:59.967430 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8d0a2ac7-9f8d-4be0-ae58-bbc7d86eec5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:37:59.970383 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8d0a2ac7-9f8d-4be0-ae58-bbc7d86eec5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:37:59.971772 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8d0a2ac7-9f8d-4be0-ae58-bbc7d86eec5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:00.230595 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8d0a2ac7-9f8d-4be0-ae58-bbc7d86eec5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:00.239623 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=100 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:00.241204 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8d0a2ac7-9f8d-4be0-ae58-bbc7d86eec5f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1510 microversion: 2.1 time: 0.312423 May 28 03:38:00.242120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 129/273] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:37:59 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1510 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:00.349301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8132109a-b47e-4e55-8eeb-42ed6ec6cc2d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:00.351151 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8132109a-b47e-4e55-8eeb-42ed6ec6cc2d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:00.358896 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8132109a-b47e-4e55-8eeb-42ed6ec6cc2d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:00.359356 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8132109a-b47e-4e55-8eeb-42ed6ec6cc2d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:00.360185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8132109a-b47e-4e55-8eeb-42ed6ec6cc2d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:00.360704 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8132109a-b47e-4e55-8eeb-42ed6ec6cc2d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:00.424764 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6c6b6c1b-082f-4bb6-9c7f-881c34848464 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:00.425401 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6c6b6c1b-082f-4bb6-9c7f-881c34848464 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:00.455193 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6c6b6c1b-082f-4bb6-9c7f-881c34848464 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:00.455669 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6c6b6c1b-082f-4bb6-9c7f-881c34848464 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:00.461655 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6c6b6c1b-082f-4bb6-9c7f-881c34848464 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:00.462120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6c6b6c1b-082f-4bb6-9c7f-881c34848464 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:00.641421 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8132109a-b47e-4e55-8eeb-42ed6ec6cc2d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:00.656729 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8132109a-b47e-4e55-8eeb-42ed6ec6cc2d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:00.658825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8132109a-b47e-4e55-8eeb-42ed6ec6cc2d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:00.666013 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8132109a-b47e-4e55-8eeb-42ed6ec6cc2d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.315674 May 28 03:38:00.666013 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 145/274] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:00 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:38:00.666830 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-18913e44-cb57-4e63-96a1-49450bf077ab tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:00.667897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-18913e44-cb57-4e63-96a1-49450bf077ab tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:00.674950 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-18913e44-cb57-4e63-96a1-49450bf077ab tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:00.675710 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-18913e44-cb57-4e63-96a1-49450bf077ab tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:00.677081 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-18913e44-cb57-4e63-96a1-49450bf077ab tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:00.677545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-18913e44-cb57-4e63-96a1-49450bf077ab tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:00.770299 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6c6b6c1b-082f-4bb6-9c7f-881c34848464 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:00.776086 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6c6b6c1b-082f-4bb6-9c7f-881c34848464 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.356246 May 28 03:38:00.782773 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 130/275] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:00 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:00.799392 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-18913e44-cb57-4e63-96a1-49450bf077ab tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:00.810273 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-18913e44-cb57-4e63-96a1-49450bf077ab tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1292 microversion: 2.1 time: 0.147180 May 28 03:38:00.820500 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 146/276] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:00 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1292 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:01.082474 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-840c7c7f-e3cb-4410-930f-4994e1cc4fe8 req-5c651819-821c-49fa-a985-bead4db3d0c8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f966dbeb-97e6-4540-acd6-e64e52e4e45b", "tag": "ffc62fe3-a4d5-4cba-b924-8fbde57f6682"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:01.097211 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-840c7c7f-e3cb-4410-930f-4994e1cc4fe8 req-5c651819-821c-49fa-a985-bead4db3d0c8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:01.102924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-840c7c7f-e3cb-4410-930f-4994e1cc4fe8 req-5c651819-821c-49fa-a985-bead4db3d0c8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:01.126923 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-840c7c7f-e3cb-4410-930f-4994e1cc4fe8 req-5c651819-821c-49fa-a985-bead4db3d0c8 service nova] Creating event network-changed:ffc62fe3-a4d5-4cba-b924-8fbde57f6682 for instance f966dbeb-97e6-4540-acd6-e64e52e4e45b on hv-cinder-84336 May 28 03:38:01.139264 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-840c7c7f-e3cb-4410-930f-4994e1cc4fe8 req-5c651819-821c-49fa-a985-bead4db3d0c8 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060370 May 28 03:38:01.143718 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 131/277] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:01.260014 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-088acb40-ee34-4695-8904-6b841f211fdd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:01.260670 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-088acb40-ee34-4695-8904-6b841f211fdd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:01.270891 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-088acb40-ee34-4695-8904-6b841f211fdd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:01.272826 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-088acb40-ee34-4695-8904-6b841f211fdd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:01.273695 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-088acb40-ee34-4695-8904-6b841f211fdd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:01.274204 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-088acb40-ee34-4695-8904-6b841f211fdd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:01.486139 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-088acb40-ee34-4695-8904-6b841f211fdd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:01.512976 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-088acb40-ee34-4695-8904-6b841f211fdd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1510 microversion: 2.1 time: 0.253262 May 28 03:38:01.513857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 147/278] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:01 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1510 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:01.688054 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-080da663-22ce-4e2a-8d67-a1cf181f55ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:01.688054 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-080da663-22ce-4e2a-8d67-a1cf181f55ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:01.696670 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-080da663-22ce-4e2a-8d67-a1cf181f55ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:01.698679 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-080da663-22ce-4e2a-8d67-a1cf181f55ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:01.700672 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-080da663-22ce-4e2a-8d67-a1cf181f55ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:01.702692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-080da663-22ce-4e2a-8d67-a1cf181f55ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:01.795494 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bd5cd771-98fd-4ab7-95da-f90e25e23725 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:01.796552 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bd5cd771-98fd-4ab7-95da-f90e25e23725 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:01.806837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd5cd771-98fd-4ab7-95da-f90e25e23725 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:01.811317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd5cd771-98fd-4ab7-95da-f90e25e23725 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:01.812711 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd5cd771-98fd-4ab7-95da-f90e25e23725 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:01.812981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd5cd771-98fd-4ab7-95da-f90e25e23725 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:01.928419 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-080da663-22ce-4e2a-8d67-a1cf181f55ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:01.944352 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-080da663-22ce-4e2a-8d67-a1cf181f55ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:01.946217 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-080da663-22ce-4e2a-8d67-a1cf181f55ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:01.947470 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-080da663-22ce-4e2a-8d67-a1cf181f55ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.266439 May 28 03:38:01.957992 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 132/279] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:01 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:01.962433 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-650ad3b9-e33e-4b2f-81a5-a55b35c7e786 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:01.963192 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-650ad3b9-e33e-4b2f-81a5-a55b35c7e786 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:01.970445 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-650ad3b9-e33e-4b2f-81a5-a55b35c7e786 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:01.970908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-650ad3b9-e33e-4b2f-81a5-a55b35c7e786 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:01.971760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-650ad3b9-e33e-4b2f-81a5-a55b35c7e786 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:01.972205 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-650ad3b9-e33e-4b2f-81a5-a55b35c7e786 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:02.063980 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bd5cd771-98fd-4ab7-95da-f90e25e23725 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:02.073464 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bd5cd771-98fd-4ab7-95da-f90e25e23725 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.282585 May 28 03:38:02.074045 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 148/280] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:01 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:02.089076 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-650ad3b9-e33e-4b2f-81a5-a55b35c7e786 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:02.100477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-650ad3b9-e33e-4b2f-81a5-a55b35c7e786 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1292 microversion: 2.1 time: 0.150974 May 28 03:38:02.102649 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 133/281] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:01 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1292 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:02.551584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3ea91fde-9ec3-481c-94d0-23b11fbf285c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:02.551584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3ea91fde-9ec3-481c-94d0-23b11fbf285c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:02.558712 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ea91fde-9ec3-481c-94d0-23b11fbf285c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:02.559416 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ea91fde-9ec3-481c-94d0-23b11fbf285c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:02.560835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ea91fde-9ec3-481c-94d0-23b11fbf285c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:02.561473 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ea91fde-9ec3-481c-94d0-23b11fbf285c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:02.716866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3ea91fde-9ec3-481c-94d0-23b11fbf285c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:02.723891 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3ea91fde-9ec3-481c-94d0-23b11fbf285c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1510 microversion: 2.1 time: 0.185652 May 28 03:38:02.724472 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 149/282] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:02 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1510 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:02.965884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2f36821d-2cc2-4740-946c-c89a43f9f6a8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:02.966446 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2f36821d-2cc2-4740-946c-c89a43f9f6a8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:02.976233 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2f36821d-2cc2-4740-946c-c89a43f9f6a8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:02.976718 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2f36821d-2cc2-4740-946c-c89a43f9f6a8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:02.977554 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2f36821d-2cc2-4740-946c-c89a43f9f6a8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:02.977993 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2f36821d-2cc2-4740-946c-c89a43f9f6a8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:03.089869 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d9dfb335-6933-4061-8e93-99f006a29cef tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:03.090794 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d9dfb335-6933-4061-8e93-99f006a29cef tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:03.097774 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9dfb335-6933-4061-8e93-99f006a29cef tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:03.098219 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9dfb335-6933-4061-8e93-99f006a29cef tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:03.099146 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9dfb335-6933-4061-8e93-99f006a29cef tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:03.099632 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9dfb335-6933-4061-8e93-99f006a29cef tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:03.330341 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2f36821d-2cc2-4740-946c-c89a43f9f6a8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:03.347622 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2f36821d-2cc2-4740-946c-c89a43f9f6a8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:03.349789 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2f36821d-2cc2-4740-946c-c89a43f9f6a8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:03.350983 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2f36821d-2cc2-4740-946c-c89a43f9f6a8 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.389025 May 28 03:38:03.351735 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 134/283] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:02 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:03.356376 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-20ab8d3d-d227-4a05-83c6-743873576787 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:03.367546 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-20ab8d3d-d227-4a05-83c6-743873576787 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:03.378490 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d9dfb335-6933-4061-8e93-99f006a29cef tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:03.383060 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20ab8d3d-d227-4a05-83c6-743873576787 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:03.387946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20ab8d3d-d227-4a05-83c6-743873576787 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:03.389650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20ab8d3d-d227-4a05-83c6-743873576787 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:03.391641 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20ab8d3d-d227-4a05-83c6-743873576787 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:03.396459 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d9dfb335-6933-4061-8e93-99f006a29cef tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.308442 May 28 03:38:03.397603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 150/284] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:03 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:03.504442 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-20ab8d3d-d227-4a05-83c6-743873576787 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:03.529315 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-20ab8d3d-d227-4a05-83c6-743873576787 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1300 microversion: 2.1 time: 0.176203 May 28 03:38:03.530597 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 135/285] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:03 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1300 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:03.746198 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1f527a01-9e33-4e90-9598-8c41b4a44fbb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:03.746819 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1f527a01-9e33-4e90-9598-8c41b4a44fbb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:03.759980 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f527a01-9e33-4e90-9598-8c41b4a44fbb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:03.762708 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f527a01-9e33-4e90-9598-8c41b4a44fbb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:03.765120 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f527a01-9e33-4e90-9598-8c41b4a44fbb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:03.767161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f527a01-9e33-4e90-9598-8c41b4a44fbb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:03.928208 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3cb61ad6-5fbd-4b12-887e-b6019b537a73 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:03.941290 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3cb61ad6-5fbd-4b12-887e-b6019b537a73 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:03.946011 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3cb61ad6-5fbd-4b12-887e-b6019b537a73 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:04.044309 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3cb61ad6-5fbd-4b12-887e-b6019b537a73 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Going to try to stop instance {{(pid=124605) force_stop /opt/stack/nova/nova/compute/api.py:2707}} May 28 03:38:04.094641 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1f527a01-9e33-4e90-9598-8c41b4a44fbb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:04.108804 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1f527a01-9e33-4e90-9598-8c41b4a44fbb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1516 microversion: 2.1 time: 0.368467 May 28 03:38:04.115071 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 151/286] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:03 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1516 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:04.310319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3cb61ad6-5fbd-4b12-887e-b6019b537a73 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "POST /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/action" status: 202 len: 0 microversion: 2.1 time: 0.388634 May 28 03:38:04.310991 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 136/287] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:38:03 2022] POST /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/action => generated 0 bytes in 390 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:38:04.320130 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-06f221cf-4c49-4241-9c94-2e33ca5939b4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:04.321002 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-06f221cf-4c49-4241-9c94-2e33ca5939b4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:04.331037 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06f221cf-4c49-4241-9c94-2e33ca5939b4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:04.334307 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06f221cf-4c49-4241-9c94-2e33ca5939b4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:04.334307 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06f221cf-4c49-4241-9c94-2e33ca5939b4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:04.334307 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06f221cf-4c49-4241-9c94-2e33ca5939b4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:04.370910 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1b35aebf-439f-405c-8df3-eb19011ba095 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:04.371599 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1b35aebf-439f-405c-8df3-eb19011ba095 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:04.380637 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1b35aebf-439f-405c-8df3-eb19011ba095 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:04.386288 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1b35aebf-439f-405c-8df3-eb19011ba095 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:04.389310 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1b35aebf-439f-405c-8df3-eb19011ba095 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:04.389310 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1b35aebf-439f-405c-8df3-eb19011ba095 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:04.572713 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-06f221cf-4c49-4241-9c94-2e33ca5939b4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:04.584104 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-06f221cf-4c49-4241-9c94-2e33ca5939b4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.268294 May 28 03:38:04.585598 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 152/288] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:04 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:04.591302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f70053d7-1278-46d0-921d-a7912338142a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:04.593484 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f70053d7-1278-46d0-921d-a7912338142a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:04.600070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f70053d7-1278-46d0-921d-a7912338142a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:04.600794 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f70053d7-1278-46d0-921d-a7912338142a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:04.602645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f70053d7-1278-46d0-921d-a7912338142a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:04.604302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f70053d7-1278-46d0-921d-a7912338142a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:04.683146 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1b35aebf-439f-405c-8df3-eb19011ba095 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:04.691777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1b35aebf-439f-405c-8df3-eb19011ba095 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:04.693593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1b35aebf-439f-405c-8df3-eb19011ba095 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:04.694985 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1b35aebf-439f-405c-8df3-eb19011ba095 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.328331 May 28 03:38:04.705591 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 137/289] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:04 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:38:04.711886 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f9494094-5de8-45d2-8b06-aeea821f8593 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:04.712663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f9494094-5de8-45d2-8b06-aeea821f8593 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:04.719682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f9494094-5de8-45d2-8b06-aeea821f8593 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:04.720354 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f9494094-5de8-45d2-8b06-aeea821f8593 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:04.721324 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f9494094-5de8-45d2-8b06-aeea821f8593 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:04.721901 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f9494094-5de8-45d2-8b06-aeea821f8593 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:04.806761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f9494094-5de8-45d2-8b06-aeea821f8593 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:04.819233 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f9494094-5de8-45d2-8b06-aeea821f8593 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1298 microversion: 2.1 time: 0.112213 May 28 03:38:04.832916 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 138/290] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:04 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1298 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:04.877368 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f70053d7-1278-46d0-921d-a7912338142a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:04.886873 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f70053d7-1278-46d0-921d-a7912338142a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.299988 May 28 03:38:04.898909 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 153/291] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:04 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:05.606129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-10c8c74c-e331-4619-9149-e085b1f683fc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:05.607495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-10c8c74c-e331-4619-9149-e085b1f683fc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:05.616317 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-10c8c74c-e331-4619-9149-e085b1f683fc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:05.616896 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-10c8c74c-e331-4619-9149-e085b1f683fc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:05.617767 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-10c8c74c-e331-4619-9149-e085b1f683fc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:05.618431 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-10c8c74c-e331-4619-9149-e085b1f683fc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:05.715858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-946e03da-17fd-4ba4-87af-e0c03d373db4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:05.716487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-946e03da-17fd-4ba4-87af-e0c03d373db4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:05.728613 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-946e03da-17fd-4ba4-87af-e0c03d373db4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:05.729411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-946e03da-17fd-4ba4-87af-e0c03d373db4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:05.730389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-946e03da-17fd-4ba4-87af-e0c03d373db4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:05.732642 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-946e03da-17fd-4ba4-87af-e0c03d373db4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:05.839149 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-10c8c74c-e331-4619-9149-e085b1f683fc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:05.853937 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-10c8c74c-e331-4619-9149-e085b1f683fc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.254369 May 28 03:38:05.865248 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 139/292] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:05 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:05.870988 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3ab6c5c9-b4ba-4d01-878d-736661ee836a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:05.873398 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3ab6c5c9-b4ba-4d01-878d-736661ee836a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:05.896110 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ab6c5c9-b4ba-4d01-878d-736661ee836a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:05.896567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ab6c5c9-b4ba-4d01-878d-736661ee836a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:05.900781 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ab6c5c9-b4ba-4d01-878d-736661ee836a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:05.904361 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ab6c5c9-b4ba-4d01-878d-736661ee836a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:06.025617 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-946e03da-17fd-4ba4-87af-e0c03d373db4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:06.035599 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-946e03da-17fd-4ba4-87af-e0c03d373db4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:06.038427 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-946e03da-17fd-4ba4-87af-e0c03d373db4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:06.039853 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-946e03da-17fd-4ba4-87af-e0c03d373db4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.327891 May 28 03:38:06.051403 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 154/293] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:05 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:38:06.056807 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-324fc71b-cdcb-4400-b636-a415805566d0 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:06.057455 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-324fc71b-cdcb-4400-b636-a415805566d0 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:06.064395 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-324fc71b-cdcb-4400-b636-a415805566d0 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:06.065715 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-324fc71b-cdcb-4400-b636-a415805566d0 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:06.067216 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-324fc71b-cdcb-4400-b636-a415805566d0 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:06.067816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-324fc71b-cdcb-4400-b636-a415805566d0 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:06.144821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3ab6c5c9-b4ba-4d01-878d-736661ee836a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:06.152865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3ab6c5c9-b4ba-4d01-878d-736661ee836a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.286383 May 28 03:38:06.162821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 140/294] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:05 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:06.371857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-324fc71b-cdcb-4400-b636-a415805566d0 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:06.383256 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-324fc71b-cdcb-4400-b636-a415805566d0 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.330691 May 28 03:38:06.385024 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 155/295] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:06 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:06.880259 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d69bed5b-4eb2-42ee-854c-daf73ee96591 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:06.881019 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d69bed5b-4eb2-42ee-854c-daf73ee96591 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:06.887922 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d69bed5b-4eb2-42ee-854c-daf73ee96591 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:06.888489 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d69bed5b-4eb2-42ee-854c-daf73ee96591 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:06.889376 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d69bed5b-4eb2-42ee-854c-daf73ee96591 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:06.889904 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d69bed5b-4eb2-42ee-854c-daf73ee96591 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:07.040157 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d69bed5b-4eb2-42ee-854c-daf73ee96591 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:07.048802 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d69bed5b-4eb2-42ee-854c-daf73ee96591 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.172978 May 28 03:38:07.109084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 141/296] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:06 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:07.109259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4a9aba82-608e-4445-a10c-8df4e7f7960b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:07.109259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4a9aba82-608e-4445-a10c-8df4e7f7960b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:07.109259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a9aba82-608e-4445-a10c-8df4e7f7960b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:07.109259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a9aba82-608e-4445-a10c-8df4e7f7960b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:07.109889 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a9aba82-608e-4445-a10c-8df4e7f7960b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:07.109889 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a9aba82-608e-4445-a10c-8df4e7f7960b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:07.170520 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ab9144c3-6a2a-441e-9043-ea179e09e004 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:07.171481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ab9144c3-6a2a-441e-9043-ea179e09e004 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:07.178368 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ab9144c3-6a2a-441e-9043-ea179e09e004 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:07.178826 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ab9144c3-6a2a-441e-9043-ea179e09e004 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:07.179709 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ab9144c3-6a2a-441e-9043-ea179e09e004 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:07.180315 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ab9144c3-6a2a-441e-9043-ea179e09e004 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:07.218602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4a9aba82-608e-4445-a10c-8df4e7f7960b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:07.227459 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4a9aba82-608e-4445-a10c-8df4e7f7960b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:07.229083 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4a9aba82-608e-4445-a10c-8df4e7f7960b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:07.230278 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4a9aba82-608e-4445-a10c-8df4e7f7960b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.176262 May 28 03:38:07.237447 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 156/297] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:07 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:38:07.329701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ab9144c3-6a2a-441e-9043-ea179e09e004 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:07.338572 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ab9144c3-6a2a-441e-9043-ea179e09e004 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.171875 May 28 03:38:07.339173 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 142/298] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:07 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:07.400713 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-84533cc9-142b-4e14-b716-998776143855 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:07.401271 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-84533cc9-142b-4e14-b716-998776143855 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:07.407273 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-84533cc9-142b-4e14-b716-998776143855 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:07.407786 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-84533cc9-142b-4e14-b716-998776143855 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:07.408471 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-84533cc9-142b-4e14-b716-998776143855 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:07.408900 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-84533cc9-142b-4e14-b716-998776143855 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:07.439181 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:07.568380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-84533cc9-142b-4e14-b716-998776143855 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:07.591101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-84533cc9-142b-4e14-b716-998776143855 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.194049 May 28 03:38:07.591733 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 157/299] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:07 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:08.062012 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0b9a0070-c4a1-4fca-9c28-2b83f3c1c1e0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Action: 'create', calling method: .version_select at 0x7feb1fef9d30>, body: {"keypair": {"name": "tempest-keypair-133479696", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFXrijosJRWyJd8qapf8t3kkjQZriOUxLhmsYFmkEGf2c8YcS05H6l0PT4gGXO7p7h4NFlqOBxhcbjR2avHU0wG5MO5/cfd78FBCjp5caJGyGhHVOYQrKgRwzcITIQ0jjQ=="}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:08.069259 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:08.078489 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-0b9a0070-c4a1-4fca-9c28-2b83f3c1c1e0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Getting quotas for project 5cab8255c6134aebb9c93493a2342cb1. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:38:08.084106 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-0b9a0070-c4a1-4fca-9c28-2b83f3c1c1e0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Getting quotas for user b6709eed72ca46468fd298929ae0f764 and project 5cab8255c6134aebb9c93493a2342cb1. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:38:08.128522 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-69e67380-6948-4ed7-ae86-68867b426719 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:08.129221 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-69e67380-6948-4ed7-ae86-68867b426719 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:08.137774 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=4 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:08.149554 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=14 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:08.152283 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:08.155404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-69e67380-6948-4ed7-ae86-68867b426719 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:08.155854 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-69e67380-6948-4ed7-ae86-68867b426719 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:08.156645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-69e67380-6948-4ed7-ae86-68867b426719 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:08.157065 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-69e67380-6948-4ed7-ae86-68867b426719 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:08.178789 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0b9a0070-c4a1-4fca-9c28-2b83f3c1c1e0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.120789 May 28 03:38:08.182543 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 143/300] 10.222.0.41 () {60 vars in 1158 bytes} [Sat May 28 03:38:08 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:08.250053 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f3ca31d6-af4d-411a-98e8-d93bfabcba7d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:08.250654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f3ca31d6-af4d-411a-98e8-d93bfabcba7d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:08.261737 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3ca31d6-af4d-411a-98e8-d93bfabcba7d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:08.262187 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3ca31d6-af4d-411a-98e8-d93bfabcba7d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:08.273582 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3ca31d6-af4d-411a-98e8-d93bfabcba7d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:08.274333 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3ca31d6-af4d-411a-98e8-d93bfabcba7d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:08.485904 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-69e67380-6948-4ed7-ae86-68867b426719 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:08.501838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-69e67380-6948-4ed7-ae86-68867b426719 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.381344 May 28 03:38:08.504614 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 158/301] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:08 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:08.515013 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8475dc6d-6a3b-483c-a0e2-6865831b2d00 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:08.516662 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8475dc6d-6a3b-483c-a0e2-6865831b2d00 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:08.527848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8475dc6d-6a3b-483c-a0e2-6865831b2d00 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:08.533335 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8475dc6d-6a3b-483c-a0e2-6865831b2d00 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:08.534387 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8475dc6d-6a3b-483c-a0e2-6865831b2d00 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:08.534978 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8475dc6d-6a3b-483c-a0e2-6865831b2d00 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:08.666904 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f3ca31d6-af4d-411a-98e8-d93bfabcba7d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:08.680263 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f3ca31d6-af4d-411a-98e8-d93bfabcba7d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:08.682525 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f3ca31d6-af4d-411a-98e8-d93bfabcba7d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:08.691025 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f3ca31d6-af4d-411a-98e8-d93bfabcba7d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.444144 May 28 03:38:08.693185 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 144/302] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:08 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:08.710703 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-bd327469-2d2b-4db1-89bf-295da02e1730 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:08.711537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-bd327469-2d2b-4db1-89bf-295da02e1730 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:08.727064 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bd327469-2d2b-4db1-89bf-295da02e1730 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:08.727616 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bd327469-2d2b-4db1-89bf-295da02e1730 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:08.729163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bd327469-2d2b-4db1-89bf-295da02e1730 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:08.730905 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bd327469-2d2b-4db1-89bf-295da02e1730 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:08.772923 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8475dc6d-6a3b-483c-a0e2-6865831b2d00 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:08.793713 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8475dc6d-6a3b-483c-a0e2-6865831b2d00 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.287705 May 28 03:38:08.794643 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 159/303] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:08 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:08.938995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-bd327469-2d2b-4db1-89bf-295da02e1730 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:08.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-bd327469-2d2b-4db1-89bf-295da02e1730 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.249324 May 28 03:38:08.949227 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 145/304] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:08 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:09.435641 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-86110f88-1bcb-444f-a9b3-17ad547fe6f7 req-0e0978c6-fc60-4722-8b1d-2f0c962f11ea service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8714c46b-f821-4c0a-b55d-1083da5eb5b8", "tag": "bce2d221-cb90-4b9d-b2ca-1b1020b1c795"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:09.456104 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-86110f88-1bcb-444f-a9b3-17ad547fe6f7 req-0e0978c6-fc60-4722-8b1d-2f0c962f11ea service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:09.457698 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-86110f88-1bcb-444f-a9b3-17ad547fe6f7 req-0e0978c6-fc60-4722-8b1d-2f0c962f11ea service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:09.498176 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-86110f88-1bcb-444f-a9b3-17ad547fe6f7 req-0e0978c6-fc60-4722-8b1d-2f0c962f11ea service nova] Creating event network-changed:bce2d221-cb90-4b9d-b2ca-1b1020b1c795 for instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 on hv-cinder-84336 May 28 03:38:09.508002 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-86110f88-1bcb-444f-a9b3-17ad547fe6f7 req-0e0978c6-fc60-4722-8b1d-2f0c962f11ea service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084512 May 28 03:38:09.511231 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 160/305] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:09.525554 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0d73676c-c7f8-4014-96e9-fb5eb4438a9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:09.526736 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0d73676c-c7f8-4014-96e9-fb5eb4438a9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:09.533431 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d73676c-c7f8-4014-96e9-fb5eb4438a9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:09.534807 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d73676c-c7f8-4014-96e9-fb5eb4438a9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:09.536396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d73676c-c7f8-4014-96e9-fb5eb4438a9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:09.537028 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d73676c-c7f8-4014-96e9-fb5eb4438a9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:09.721524 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a17d9023-a556-4dca-ae9f-02fa55257356 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:09.722234 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a17d9023-a556-4dca-ae9f-02fa55257356 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:09.735931 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a17d9023-a556-4dca-ae9f-02fa55257356 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:09.735931 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a17d9023-a556-4dca-ae9f-02fa55257356 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:09.746278 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a17d9023-a556-4dca-ae9f-02fa55257356 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:09.747099 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a17d9023-a556-4dca-ae9f-02fa55257356 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:09.869586 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0d73676c-c7f8-4014-96e9-fb5eb4438a9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:09.880249 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0d73676c-c7f8-4014-96e9-fb5eb4438a9f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.361960 May 28 03:38:09.887593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 146/306] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:09 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:09.900900 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c5e1de46-ebb6-4b50-8db8-b4868b3cd672 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:09.901544 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c5e1de46-ebb6-4b50-8db8-b4868b3cd672 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:09.931393 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c5e1de46-ebb6-4b50-8db8-b4868b3cd672 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:09.932691 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c5e1de46-ebb6-4b50-8db8-b4868b3cd672 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:09.936980 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c5e1de46-ebb6-4b50-8db8-b4868b3cd672 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:09.937457 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c5e1de46-ebb6-4b50-8db8-b4868b3cd672 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:09.946987 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:10.080086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a17d9023-a556-4dca-ae9f-02fa55257356 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:10.093605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a17d9023-a556-4dca-ae9f-02fa55257356 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:10.096771 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a17d9023-a556-4dca-ae9f-02fa55257356 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:10.103060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a17d9023-a556-4dca-ae9f-02fa55257356 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.385857 May 28 03:38:10.106155 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 161/307] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:09 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:38:10.117560 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b903c363-ece1-4e62-abb8-f1eb4c88c632 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:10.118507 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b903c363-ece1-4e62-abb8-f1eb4c88c632 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:10.131622 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b903c363-ece1-4e62-abb8-f1eb4c88c632 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:10.132071 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b903c363-ece1-4e62-abb8-f1eb4c88c632 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:10.132847 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b903c363-ece1-4e62-abb8-f1eb4c88c632 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:10.133334 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b903c363-ece1-4e62-abb8-f1eb4c88c632 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:10.245144 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c5e1de46-ebb6-4b50-8db8-b4868b3cd672 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:10.258941 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c5e1de46-ebb6-4b50-8db8-b4868b3cd672 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.369752 May 28 03:38:10.259530 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 147/308] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:09 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:10.413289 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b903c363-ece1-4e62-abb8-f1eb4c88c632 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:10.421576 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b903c363-ece1-4e62-abb8-f1eb4c88c632 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.314130 May 28 03:38:10.426849 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 162/309] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:10 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:10.902264 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-137b1640-79a0-4ead-a664-dd7ade6d5fcf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:10.903594 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-137b1640-79a0-4ead-a664-dd7ade6d5fcf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:10.920608 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-137b1640-79a0-4ead-a664-dd7ade6d5fcf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:10.921742 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-137b1640-79a0-4ead-a664-dd7ade6d5fcf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:10.923700 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-137b1640-79a0-4ead-a664-dd7ade6d5fcf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:10.927654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-137b1640-79a0-4ead-a664-dd7ade6d5fcf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:11.091935 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-137b1640-79a0-4ead-a664-dd7ade6d5fcf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:11.102092 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-137b1640-79a0-4ead-a664-dd7ade6d5fcf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.205724 May 28 03:38:11.103277 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 148/310] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:10 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:11.129732 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-52eb3786-018e-4177-82b2-3c254fa95508 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:11.130957 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-52eb3786-018e-4177-82b2-3c254fa95508 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:11.149480 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-52eb3786-018e-4177-82b2-3c254fa95508 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:11.150520 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-52eb3786-018e-4177-82b2-3c254fa95508 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:11.154264 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-52eb3786-018e-4177-82b2-3c254fa95508 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:11.166474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-52eb3786-018e-4177-82b2-3c254fa95508 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:11.276665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f4f8c9df-6779-4f53-a685-a2bac4eae6bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:11.277305 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f4f8c9df-6779-4f53-a685-a2bac4eae6bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:11.286590 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f4f8c9df-6779-4f53-a685-a2bac4eae6bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:11.287028 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f4f8c9df-6779-4f53-a685-a2bac4eae6bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:11.292168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f4f8c9df-6779-4f53-a685-a2bac4eae6bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:11.292798 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f4f8c9df-6779-4f53-a685-a2bac4eae6bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:11.508835 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f4f8c9df-6779-4f53-a685-a2bac4eae6bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:11.526384 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f4f8c9df-6779-4f53-a685-a2bac4eae6bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.253095 May 28 03:38:11.528921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 149/311] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:11 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:11.539374 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f1dea3f2-3cba-4ff0-a78c-af1443289a9d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:11.539374 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f1dea3f2-3cba-4ff0-a78c-af1443289a9d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:11.547272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1dea3f2-3cba-4ff0-a78c-af1443289a9d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:11.550811 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1dea3f2-3cba-4ff0-a78c-af1443289a9d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:11.550811 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1dea3f2-3cba-4ff0-a78c-af1443289a9d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:11.550811 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1dea3f2-3cba-4ff0-a78c-af1443289a9d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:11.603941 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-52eb3786-018e-4177-82b2-3c254fa95508 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:11.634351 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-52eb3786-018e-4177-82b2-3c254fa95508 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:11.639512 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-52eb3786-018e-4177-82b2-3c254fa95508 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:11.641938 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-52eb3786-018e-4177-82b2-3c254fa95508 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.517854 May 28 03:38:11.645325 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 163/312] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:11 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:11.658822 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1144911105", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a3076c7d-656d-4e5b-8611-4fd5569b4e24"}], "security_groups": [{"name": "tempest-securitygroup--1497417973"}], "key_name": "tempest-keypair-133479696", "user_data": "}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:11.825187 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] validate_networks() for [('a3076c7d-656d-4e5b-8611-4fd5569b4e24', None, None, None, None, None)] {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:38:11.858235 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f1dea3f2-3cba-4ff0-a78c-af1443289a9d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:11.872558 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f1dea3f2-3cba-4ff0-a78c-af1443289a9d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.343169 May 28 03:38:11.873536 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 150/313] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:11 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:12.126470 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-bf4b8c2b-bf0a-4c0c-a554-ef71fe3ece0d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:12.127119 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-bf4b8c2b-bf0a-4c0c-a554-ef71fe3ece0d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:12.138393 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bf4b8c2b-bf0a-4c0c-a554-ef71fe3ece0d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:12.139137 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bf4b8c2b-bf0a-4c0c-a554-ef71fe3ece0d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:12.140085 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bf4b8c2b-bf0a-4c0c-a554-ef71fe3ece0d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:12.140539 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bf4b8c2b-bf0a-4c0c-a554-ef71fe3ece0d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:12.283111 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-bf4b8c2b-bf0a-4c0c-a554-ef71fe3ece0d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:12.291712 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-bf4b8c2b-bf0a-4c0c-a554-ef71fe3ece0d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.169281 May 28 03:38:12.292276 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 151/314] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:12 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:12.386498 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:38:12.387056 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:38:12.387390 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:38:12.387798 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:38:12.388212 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:38:12.400404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:12.400854 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:12.401700 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:12.402143 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:12.423388 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Getting quotas for project 5cab8255c6134aebb9c93493a2342cb1. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:38:12.427309 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Getting quotas for user b6709eed72ca46468fd298929ae0f764 and project 5cab8255c6134aebb9c93493a2342cb1. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:38:12.438097 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:38:12.440322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:38:12.547158 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-19f69974-3a58-4ada-b72c-4f53a50a8a9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:12.547769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-19f69974-3a58-4ada-b72c-4f53a50a8a9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:12.553819 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-19f69974-3a58-4ada-b72c-4f53a50a8a9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:12.554265 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-19f69974-3a58-4ada-b72c-4f53a50a8a9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:12.555020 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-19f69974-3a58-4ada-b72c-4f53a50a8a9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:12.555456 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-19f69974-3a58-4ada-b72c-4f53a50a8a9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:12.592119 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2a070453-5f0a-4295-987e-a033be26687f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 0.945674 May 28 03:38:12.596319 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 164/315] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:38:11 2022] POST /compute/v2.1/servers => generated 409 bytes in 950 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:38:12.607393 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-740b886a-3486-4cd7-8c6c-87e743d9a70b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:12.615400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-740b886a-3486-4cd7-8c6c-87e743d9a70b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:12.695701 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-740b886a-3486-4cd7-8c6c-87e743d9a70b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:12.705172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-740b886a-3486-4cd7-8c6c-87e743d9a70b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1245 microversion: 2.1 time: 0.101390 May 28 03:38:12.714681 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 165/316] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:12 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1245 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:12.714681 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8d2758fd-ff80-4aef-b63a-8ded98e385b5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:12.714681 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8d2758fd-ff80-4aef-b63a-8ded98e385b5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:12.724103 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d2758fd-ff80-4aef-b63a-8ded98e385b5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:12.727910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d2758fd-ff80-4aef-b63a-8ded98e385b5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:12.727910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d2758fd-ff80-4aef-b63a-8ded98e385b5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:12.730673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d2758fd-ff80-4aef-b63a-8ded98e385b5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:13.088066 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8d2758fd-ff80-4aef-b63a-8ded98e385b5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:13.095960 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8d2758fd-ff80-4aef-b63a-8ded98e385b5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:13.097639 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8d2758fd-ff80-4aef-b63a-8ded98e385b5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:13.098680 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8d2758fd-ff80-4aef-b63a-8ded98e385b5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.391930 May 28 03:38:13.110665 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 166/317] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:12 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:13.110665 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e3da5f9c-e0dd-4082-9a8b-f6fe03602f31 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:13.110665 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e3da5f9c-e0dd-4082-9a8b-f6fe03602f31 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:13.111850 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e3da5f9c-e0dd-4082-9a8b-f6fe03602f31 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:13.112301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e3da5f9c-e0dd-4082-9a8b-f6fe03602f31 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:13.113054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e3da5f9c-e0dd-4082-9a8b-f6fe03602f31 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:13.118625 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e3da5f9c-e0dd-4082-9a8b-f6fe03602f31 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:13.123415 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-19f69974-3a58-4ada-b72c-4f53a50a8a9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:13.132439 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-19f69974-3a58-4ada-b72c-4f53a50a8a9d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.589249 May 28 03:38:13.137177 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 152/318] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:12 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:13.309370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-aba802ec-8da3-495b-b2f1-e6f59cef5664 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:13.310401 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-aba802ec-8da3-495b-b2f1-e6f59cef5664 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:13.331290 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-aba802ec-8da3-495b-b2f1-e6f59cef5664 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:13.332549 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-aba802ec-8da3-495b-b2f1-e6f59cef5664 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:13.333519 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-aba802ec-8da3-495b-b2f1-e6f59cef5664 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:13.333941 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-aba802ec-8da3-495b-b2f1-e6f59cef5664 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:13.381613 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e3da5f9c-e0dd-4082-9a8b-f6fe03602f31 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:13.391828 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e3da5f9c-e0dd-4082-9a8b-f6fe03602f31 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.291345 May 28 03:38:13.392414 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 167/319] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:13 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:13.591978 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-aba802ec-8da3-495b-b2f1-e6f59cef5664 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:13.605985 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-aba802ec-8da3-495b-b2f1-e6f59cef5664 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.300305 May 28 03:38:13.607293 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 153/320] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:13 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:13.647261 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-47be0d3e-cc52-4826-ac85-0eca76b6b5c6 req-ba46aa1a-45bf-4e41-9d12-9cdbe7173a4e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f966dbeb-97e6-4540-acd6-e64e52e4e45b", "name": "network-vif-plugged", "status": "completed", "tag": "ffc62fe3-a4d5-4cba-b924-8fbde57f6682"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:13.679778 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-47be0d3e-cc52-4826-ac85-0eca76b6b5c6 req-ba46aa1a-45bf-4e41-9d12-9cdbe7173a4e service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:13.680302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-47be0d3e-cc52-4826-ac85-0eca76b6b5c6 req-ba46aa1a-45bf-4e41-9d12-9cdbe7173a4e service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:13.695417 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-47be0d3e-cc52-4826-ac85-0eca76b6b5c6 req-ba46aa1a-45bf-4e41-9d12-9cdbe7173a4e service nova] Creating event network-vif-plugged:ffc62fe3-a4d5-4cba-b924-8fbde57f6682 for instance f966dbeb-97e6-4540-acd6-e64e52e4e45b on hv-cinder-84336 May 28 03:38:13.702502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-47be0d3e-cc52-4826-ac85-0eca76b6b5c6 req-ba46aa1a-45bf-4e41-9d12-9cdbe7173a4e service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058622 May 28 03:38:13.703304 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 168/321] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:13.724132 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-fd532fba-266e-4099-b2ec-b28cc2b01c9f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:13.724998 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-fd532fba-266e-4099-b2ec-b28cc2b01c9f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:13.835944 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-fd532fba-266e-4099-b2ec-b28cc2b01c9f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:13.842790 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-fd532fba-266e-4099-b2ec-b28cc2b01c9f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1245 microversion: 2.1 time: 0.123466 May 28 03:38:13.843887 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 154/322] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:13 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1245 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:14.131018 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6cfc204d-4dc5-455e-b42e-7081092245fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:14.131841 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-6cfc204d-4dc5-455e-b42e-7081092245fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:14.138295 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6cfc204d-4dc5-455e-b42e-7081092245fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:14.138963 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6cfc204d-4dc5-455e-b42e-7081092245fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:14.139802 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6cfc204d-4dc5-455e-b42e-7081092245fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:14.140313 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6cfc204d-4dc5-455e-b42e-7081092245fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:14.152172 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-22e2d8a5-238a-4bed-b7a5-804094c14053 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:14.152790 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-22e2d8a5-238a-4bed-b7a5-804094c14053 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:14.160232 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-22e2d8a5-238a-4bed-b7a5-804094c14053 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:14.160683 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-22e2d8a5-238a-4bed-b7a5-804094c14053 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:14.161480 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-22e2d8a5-238a-4bed-b7a5-804094c14053 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:14.162208 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-22e2d8a5-238a-4bed-b7a5-804094c14053 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:14.298183 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6cfc204d-4dc5-455e-b42e-7081092245fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:14.306918 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6cfc204d-4dc5-455e-b42e-7081092245fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:14.308414 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6cfc204d-4dc5-455e-b42e-7081092245fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:14.309602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6cfc204d-4dc5-455e-b42e-7081092245fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.184199 May 28 03:38:14.310504 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 169/323] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:14 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:14.336090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-22e2d8a5-238a-4bed-b7a5-804094c14053 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:14.343541 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-22e2d8a5-238a-4bed-b7a5-804094c14053 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.195625 May 28 03:38:14.344105 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 155/324] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:14 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:14.409739 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a13f94c8-08b2-4c0a-8c76-a3b7da076159 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:14.410777 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a13f94c8-08b2-4c0a-8c76-a3b7da076159 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:14.416933 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a13f94c8-08b2-4c0a-8c76-a3b7da076159 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:14.417476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a13f94c8-08b2-4c0a-8c76-a3b7da076159 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:14.418205 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a13f94c8-08b2-4c0a-8c76-a3b7da076159 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:14.419216 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a13f94c8-08b2-4c0a-8c76-a3b7da076159 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:14.571990 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a13f94c8-08b2-4c0a-8c76-a3b7da076159 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:14.579522 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a13f94c8-08b2-4c0a-8c76-a3b7da076159 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.174172 May 28 03:38:14.580083 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 170/325] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:14 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:14.629431 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5104ef23-77dd-480c-9895-5cc0477d1604 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:14.630037 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5104ef23-77dd-480c-9895-5cc0477d1604 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:14.636359 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5104ef23-77dd-480c-9895-5cc0477d1604 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:14.638361 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5104ef23-77dd-480c-9895-5cc0477d1604 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:14.638361 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5104ef23-77dd-480c-9895-5cc0477d1604 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:14.638361 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5104ef23-77dd-480c-9895-5cc0477d1604 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:14.796161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5104ef23-77dd-480c-9895-5cc0477d1604 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:14.804247 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5104ef23-77dd-480c-9895-5cc0477d1604 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.178661 May 28 03:38:14.807885 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 156/326] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:14 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:14.860631 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bb58f351-8cf9-475e-b151-960fe4b92ba6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:14.861236 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bb58f351-8cf9-475e-b151-960fe4b92ba6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:14.867720 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bb58f351-8cf9-475e-b151-960fe4b92ba6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:14.868163 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bb58f351-8cf9-475e-b151-960fe4b92ba6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:14.869233 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bb58f351-8cf9-475e-b151-960fe4b92ba6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:14.869681 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bb58f351-8cf9-475e-b151-960fe4b92ba6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:14.952880 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bb58f351-8cf9-475e-b151-960fe4b92ba6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:14.960750 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bb58f351-8cf9-475e-b151-960fe4b92ba6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1297 microversion: 2.1 time: 0.104445 May 28 03:38:14.961556 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 171/327] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:14 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1297 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:15.330747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ec02b479-3635-41bf-8286-ebb8023ebe50 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:15.334597 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ec02b479-3635-41bf-8286-ebb8023ebe50 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:15.341316 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ec02b479-3635-41bf-8286-ebb8023ebe50 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:15.341909 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ec02b479-3635-41bf-8286-ebb8023ebe50 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:15.342730 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ec02b479-3635-41bf-8286-ebb8023ebe50 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:15.343238 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ec02b479-3635-41bf-8286-ebb8023ebe50 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:15.361411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ad899d5e-c803-41d6-aa5e-88f6e70ada12 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:15.362066 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ad899d5e-c803-41d6-aa5e-88f6e70ada12 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:15.368864 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ad899d5e-c803-41d6-aa5e-88f6e70ada12 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:15.369337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ad899d5e-c803-41d6-aa5e-88f6e70ada12 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:15.370162 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ad899d5e-c803-41d6-aa5e-88f6e70ada12 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:15.370627 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ad899d5e-c803-41d6-aa5e-88f6e70ada12 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:15.510317 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ec02b479-3635-41bf-8286-ebb8023ebe50 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:15.522648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ec02b479-3635-41bf-8286-ebb8023ebe50 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:15.525986 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ec02b479-3635-41bf-8286-ebb8023ebe50 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:15.528611 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ec02b479-3635-41bf-8286-ebb8023ebe50 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1741 microversion: 2.20 time: 0.201591 May 28 03:38:15.529774 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 157/328] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:15 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1741 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:15.566080 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ad899d5e-c803-41d6-aa5e-88f6e70ada12 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:15.576408 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ad899d5e-c803-41d6-aa5e-88f6e70ada12 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.218745 May 28 03:38:15.581213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 172/329] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:15 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:15.596375 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-df8fe6a5-fdaa-4f47-8f67-09417cc5e940 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:15.597024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-df8fe6a5-fdaa-4f47-8f67-09417cc5e940 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:15.603254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-df8fe6a5-fdaa-4f47-8f67-09417cc5e940 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:15.607221 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-df8fe6a5-fdaa-4f47-8f67-09417cc5e940 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:15.607221 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-df8fe6a5-fdaa-4f47-8f67-09417cc5e940 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:15.607221 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-df8fe6a5-fdaa-4f47-8f67-09417cc5e940 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:15.752383 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-df8fe6a5-fdaa-4f47-8f67-09417cc5e940 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:15.760537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-df8fe6a5-fdaa-4f47-8f67-09417cc5e940 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1313 microversion: 2.1 time: 0.168708 May 28 03:38:15.761288 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 158/330] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:15 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1313 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:15.821890 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a2fbcb9a-3fda-49ab-8739-1f4643f83001 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:15.822503 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a2fbcb9a-3fda-49ab-8739-1f4643f83001 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:15.828623 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2fbcb9a-3fda-49ab-8739-1f4643f83001 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:15.829069 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2fbcb9a-3fda-49ab-8739-1f4643f83001 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:15.829856 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2fbcb9a-3fda-49ab-8739-1f4643f83001 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:15.830291 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2fbcb9a-3fda-49ab-8739-1f4643f83001 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:15.961266 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a2fbcb9a-3fda-49ab-8739-1f4643f83001 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:15.969533 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a2fbcb9a-3fda-49ab-8739-1f4643f83001 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.151587 May 28 03:38:15.970101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 173/331] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:15 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:15.979622 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-212beff4-38c0-4558-9207-ff77738e3e6d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:15.980440 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-212beff4-38c0-4558-9207-ff77738e3e6d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:15.986963 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-212beff4-38c0-4558-9207-ff77738e3e6d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:15.987747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-212beff4-38c0-4558-9207-ff77738e3e6d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:15.989476 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-212beff4-38c0-4558-9207-ff77738e3e6d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:15.990247 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-212beff4-38c0-4558-9207-ff77738e3e6d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:16.055723 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-212beff4-38c0-4558-9207-ff77738e3e6d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:16.063613 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-212beff4-38c0-4558-9207-ff77738e3e6d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1297 microversion: 2.1 time: 0.087731 May 28 03:38:16.064182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 159/332] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:15 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1297 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:16.553893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-586c59b6-3054-4012-bcc9-8528f8f4368e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:16.554515 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-586c59b6-3054-4012-bcc9-8528f8f4368e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:16.567488 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-586c59b6-3054-4012-bcc9-8528f8f4368e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:16.568125 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-586c59b6-3054-4012-bcc9-8528f8f4368e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:16.572581 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-586c59b6-3054-4012-bcc9-8528f8f4368e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:16.572581 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-586c59b6-3054-4012-bcc9-8528f8f4368e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:16.599564 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7138b9dd-d0f5-4fe9-b52b-1ff9e655c739 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:16.601484 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7138b9dd-d0f5-4fe9-b52b-1ff9e655c739 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:16.609496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7138b9dd-d0f5-4fe9-b52b-1ff9e655c739 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:16.610630 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7138b9dd-d0f5-4fe9-b52b-1ff9e655c739 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:16.611430 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7138b9dd-d0f5-4fe9-b52b-1ff9e655c739 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:16.611870 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7138b9dd-d0f5-4fe9-b52b-1ff9e655c739 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:16.730676 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-586c59b6-3054-4012-bcc9-8528f8f4368e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:16.739137 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-586c59b6-3054-4012-bcc9-8528f8f4368e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:16.740627 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-586c59b6-3054-4012-bcc9-8528f8f4368e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:16.741816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-586c59b6-3054-4012-bcc9-8528f8f4368e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1723 microversion: 2.20 time: 0.197901 May 28 03:38:16.742374 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 174/333] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:16 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1723 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:16.777010 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-746ebba3-c8f2-4881-b840-e7a9f3fe3614 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:16.777790 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-746ebba3-c8f2-4881-b840-e7a9f3fe3614 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:16.784060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-746ebba3-c8f2-4881-b840-e7a9f3fe3614 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:16.796609 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-746ebba3-c8f2-4881-b840-e7a9f3fe3614 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:16.796609 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-746ebba3-c8f2-4881-b840-e7a9f3fe3614 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:16.796609 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-746ebba3-c8f2-4881-b840-e7a9f3fe3614 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:16.897800 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7138b9dd-d0f5-4fe9-b52b-1ff9e655c739 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:16.906772 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7138b9dd-d0f5-4fe9-b52b-1ff9e655c739 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.311878 May 28 03:38:16.908052 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 160/334] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:16 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:16.926802 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-746ebba3-c8f2-4881-b840-e7a9f3fe3614 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:16.934584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-746ebba3-c8f2-4881-b840-e7a9f3fe3614 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1497 microversion: 2.1 time: 0.161452 May 28 03:38:16.935152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 175/335] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:16 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:16.987808 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f1b33494-59f0-4543-bc17-4ea9b542855c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:16.989220 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f1b33494-59f0-4543-bc17-4ea9b542855c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:16.996329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1b33494-59f0-4543-bc17-4ea9b542855c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:16.997079 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1b33494-59f0-4543-bc17-4ea9b542855c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:16.997889 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1b33494-59f0-4543-bc17-4ea9b542855c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:16.998327 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1b33494-59f0-4543-bc17-4ea9b542855c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:17.080437 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-fa03a94a-3340-414e-bce7-9986944c7154 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:17.081344 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-fa03a94a-3340-414e-bce7-9986944c7154 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:17.088667 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fa03a94a-3340-414e-bce7-9986944c7154 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:17.089342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fa03a94a-3340-414e-bce7-9986944c7154 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:17.090343 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fa03a94a-3340-414e-bce7-9986944c7154 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:17.090952 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fa03a94a-3340-414e-bce7-9986944c7154 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:17.163825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-fa03a94a-3340-414e-bce7-9986944c7154 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:17.171275 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-fa03a94a-3340-414e-bce7-9986944c7154 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1303 microversion: 2.1 time: 0.094804 May 28 03:38:17.171837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 176/336] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:17 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1303 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:17.180339 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f1b33494-59f0-4543-bc17-4ea9b542855c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:17.188299 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f1b33494-59f0-4543-bc17-4ea9b542855c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.204390 May 28 03:38:17.188857 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 161/337] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:16 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:17.400063 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1787202140", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "networks": [{"uuid": "c66573d2-3992-4577-bc6d-7740bb5602ab"}]}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:17.436396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-f0e95ba6-b84c-478b-8f32-2611931bc8db req-28861bbe-33e1-4ca2-aca4-89816431af4d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8714c46b-f821-4c0a-b55d-1083da5eb5b8", "name": "network-vif-plugged", "status": "completed", "tag": "bce2d221-cb90-4b9d-b2ca-1b1020b1c795"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:17.452482 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-f0e95ba6-b84c-478b-8f32-2611931bc8db req-28861bbe-33e1-4ca2-aca4-89816431af4d service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:17.453025 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-f0e95ba6-b84c-478b-8f32-2611931bc8db req-28861bbe-33e1-4ca2-aca4-89816431af4d service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:17.469241 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-f0e95ba6-b84c-478b-8f32-2611931bc8db req-28861bbe-33e1-4ca2-aca4-89816431af4d service nova] Creating event network-vif-plugged:bce2d221-cb90-4b9d-b2ca-1b1020b1c795 for instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 on hv-cinder-84336 May 28 03:38:17.469894 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] validate_networks() for [('c66573d2-3992-4577-bc6d-7740bb5602ab', None, None, None, None, None)] {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:38:17.489909 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-f0e95ba6-b84c-478b-8f32-2611931bc8db req-28861bbe-33e1-4ca2-aca4-89816431af4d service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050965 May 28 03:38:17.489909 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 162/338] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:17.768879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9cccbaea-2d94-4fd5-aec1-eed635fb46cc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:17.770671 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9cccbaea-2d94-4fd5-aec1-eed635fb46cc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:17.778795 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9cccbaea-2d94-4fd5-aec1-eed635fb46cc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:17.779832 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9cccbaea-2d94-4fd5-aec1-eed635fb46cc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:17.780971 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9cccbaea-2d94-4fd5-aec1-eed635fb46cc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:17.782537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9cccbaea-2d94-4fd5-aec1-eed635fb46cc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:17.891545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:38:17.891545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:38:17.892110 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:38:17.892319 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:38:17.892729 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:38:17.905702 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:17.906158 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:17.907153 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:17.907590 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:17.917171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:17.940464 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Getting quotas for project e302a2cd63ba46d7a6cef952ddb89242. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:38:17.945508 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Getting quotas for user af5fa544bc6440c9830d25e4ad4d5987 and project e302a2cd63ba46d7a6cef952ddb89242. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:38:17.958306 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:38:17.960570 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:38:17.984097 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9cccbaea-2d94-4fd5-aec1-eed635fb46cc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:17.993812 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9cccbaea-2d94-4fd5-aec1-eed635fb46cc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:17.995289 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9cccbaea-2d94-4fd5-aec1-eed635fb46cc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:17.996441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9cccbaea-2d94-4fd5-aec1-eed635fb46cc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1723 microversion: 2.20 time: 0.236163 May 28 03:38:17.996994 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 163/339] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:17 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1723 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:38:18.012145 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-02faa634-8637-4c00-b7d5-2e079fc5a80c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:18.012637 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-02faa634-8637-4c00-b7d5-2e079fc5a80c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:18.018600 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-02faa634-8637-4c00-b7d5-2e079fc5a80c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:18.019043 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-02faa634-8637-4c00-b7d5-2e079fc5a80c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:18.020369 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-02faa634-8637-4c00-b7d5-2e079fc5a80c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:18.020794 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-02faa634-8637-4c00-b7d5-2e079fc5a80c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:18.044591 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-02faa634-8637-4c00-b7d5-2e079fc5a80c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:38:18.063064 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-031ea47a-6833-4e13-bdc6-0f7103588213 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.669294 May 28 03:38:18.063632 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 177/340] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:38:17 2022] POST /compute/v2.1/servers => generated 380 bytes in 671 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 28 03:38:18.069024 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5723789f-e22a-4284-bf5f-4d378f52526a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:18.069579 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=100 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:18.070172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5723789f-e22a-4284-bf5f-4d378f52526a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:18.076856 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5723789f-e22a-4284-bf5f-4d378f52526a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:18.077357 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5723789f-e22a-4284-bf5f-4d378f52526a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:18.082710 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5723789f-e22a-4284-bf5f-4d378f52526a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:18.082710 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5723789f-e22a-4284-bf5f-4d378f52526a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:18.234224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5723789f-e22a-4284-bf5f-4d378f52526a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:18.243112 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5723789f-e22a-4284-bf5f-4d378f52526a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.178265 May 28 03:38:18.244089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 178/341] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:18 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:18.249262 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c4026beb-2f43-4545-b6ea-629bebf6c62d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:18.249898 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c4026beb-2f43-4545-b6ea-629bebf6c62d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:18.301764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c4026beb-2f43-4545-b6ea-629bebf6c62d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:18.308395 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c4026beb-2f43-4545-b6ea-629bebf6c62d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1219 microversion: 2.1 time: 0.063371 May 28 03:38:18.308959 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 179/342] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:18 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1219 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:18.313351 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-17394ccd-0d02-4215-8f02-dbcd76b1b870 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:18.313947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-17394ccd-0d02-4215-8f02-dbcd76b1b870 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:18.319574 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17394ccd-0d02-4215-8f02-dbcd76b1b870 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:18.320039 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17394ccd-0d02-4215-8f02-dbcd76b1b870 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:18.320775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17394ccd-0d02-4215-8f02-dbcd76b1b870 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:18.321292 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17394ccd-0d02-4215-8f02-dbcd76b1b870 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:18.358761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-02faa634-8637-4c00-b7d5-2e079fc5a80c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "DELETE /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 204 len: 0 microversion: 2.1 time: 0.349978 May 28 03:38:18.359341 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 164/343] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:38:17 2022] DELETE /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 0 bytes in 362 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:38:18.367717 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-30d0ce10-6f10-4cf9-9647-432680475a10 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:18.368421 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-30d0ce10-6f10-4cf9-9647-432680475a10 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:18.374735 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-30d0ce10-6f10-4cf9-9647-432680475a10 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:18.375189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-30d0ce10-6f10-4cf9-9647-432680475a10 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:18.375959 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-30d0ce10-6f10-4cf9-9647-432680475a10 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:18.376396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-30d0ce10-6f10-4cf9-9647-432680475a10 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:18.454217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-17394ccd-0d02-4215-8f02-dbcd76b1b870 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:18.462177 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-17394ccd-0d02-4215-8f02-dbcd76b1b870 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.152232 May 28 03:38:18.462755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 180/344] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:18 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:18.467360 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ebbb66fd-7cb2-4bbb-bd82-e11961859fc8 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:18.468148 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ebbb66fd-7cb2-4bbb-bd82-e11961859fc8 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:18.474130 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ebbb66fd-7cb2-4bbb-bd82-e11961859fc8 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:18.474578 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ebbb66fd-7cb2-4bbb-bd82-e11961859fc8 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:18.475352 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ebbb66fd-7cb2-4bbb-bd82-e11961859fc8 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:18.475796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ebbb66fd-7cb2-4bbb-bd82-e11961859fc8 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:18.513844 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-30d0ce10-6f10-4cf9-9647-432680475a10 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:18.522306 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-30d0ce10-6f10-4cf9-9647-432680475a10 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1723 microversion: 2.1 time: 0.158844 May 28 03:38:18.523009 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 165/345] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:18 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:18.611773 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ebbb66fd-7cb2-4bbb-bd82-e11961859fc8 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:18.619949 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ebbb66fd-7cb2-4bbb-bd82-e11961859fc8 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1503 microversion: 2.1 time: 0.156266 May 28 03:38:18.620506 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 181/346] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:18 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1503 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:19.014318 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d354acd1-491e-4970-a567-fe2d40f5f09d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:19.015072 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d354acd1-491e-4970-a567-fe2d40f5f09d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:19.021497 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d354acd1-491e-4970-a567-fe2d40f5f09d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.021946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d354acd1-491e-4970-a567-fe2d40f5f09d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.022709 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d354acd1-491e-4970-a567-fe2d40f5f09d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.023135 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d354acd1-491e-4970-a567-fe2d40f5f09d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.229954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d354acd1-491e-4970-a567-fe2d40f5f09d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:19.238867 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d354acd1-491e-4970-a567-fe2d40f5f09d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:19.240693 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d354acd1-491e-4970-a567-fe2d40f5f09d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:19.242072 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d354acd1-491e-4970-a567-fe2d40f5f09d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1723 microversion: 2.20 time: 0.231777 May 28 03:38:19.242703 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 166/347] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:19 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1723 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:38:19.259786 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-989c32b3-c22c-4bdf-b55f-fdd47f7367b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:19.260626 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-989c32b3-c22c-4bdf-b55f-fdd47f7367b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:19.267224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-989c32b3-c22c-4bdf-b55f-fdd47f7367b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.267669 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-989c32b3-c22c-4bdf-b55f-fdd47f7367b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.268436 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-989c32b3-c22c-4bdf-b55f-fdd47f7367b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.268866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-989c32b3-c22c-4bdf-b55f-fdd47f7367b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.324976 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5e268511-5e61-4eb1-83ea-d6d9c917f452 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:19.325687 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5e268511-5e61-4eb1-83ea-d6d9c917f452 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:19.331822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e268511-5e61-4eb1-83ea-d6d9c917f452 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.332869 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e268511-5e61-4eb1-83ea-d6d9c917f452 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.333776 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e268511-5e61-4eb1-83ea-d6d9c917f452 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.334210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e268511-5e61-4eb1-83ea-d6d9c917f452 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.410045 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-989c32b3-c22c-4bdf-b55f-fdd47f7367b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:19.419502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-989c32b3-c22c-4bdf-b55f-fdd47f7367b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.161950 May 28 03:38:19.419502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 182/348] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:19 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:19.436843 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5e268511-5e61-4eb1-83ea-d6d9c917f452 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:19.446337 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5e268511-5e61-4eb1-83ea-d6d9c917f452 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1223 microversion: 2.1 time: 0.125231 May 28 03:38:19.446920 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 167/349] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:19 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1223 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:19.479655 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d4ff1c64-f430-46c3-932f-f45b4e2ca682 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:19.480534 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d4ff1c64-f430-46c3-932f-f45b4e2ca682 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:19.487409 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4ff1c64-f430-46c3-932f-f45b4e2ca682 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.487863 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4ff1c64-f430-46c3-932f-f45b4e2ca682 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.489445 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4ff1c64-f430-46c3-932f-f45b4e2ca682 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.489893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4ff1c64-f430-46c3-932f-f45b4e2ca682 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.498647 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-9e93ae55-85f6-43f4-ae1c-408d0b39beab req-447927de-bee1-48b6-a56a-e1c094a56a77 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35a9c645-590f-49e1-ab03-af4167f10547", "name": "network-vif-unplugged", "status": "completed", "tag": "97df7472-c1e5-4bbd-80b5-6efb1965aeb9"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:19.508123 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-9e93ae55-85f6-43f4-ae1c-408d0b39beab req-447927de-bee1-48b6-a56a-e1c094a56a77 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.508876 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-9e93ae55-85f6-43f4-ae1c-408d0b39beab req-447927de-bee1-48b6-a56a-e1c094a56a77 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.521479 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-9e93ae55-85f6-43f4-ae1c-408d0b39beab req-447927de-bee1-48b6-a56a-e1c094a56a77 service nova] Creating event network-vif-unplugged:97df7472-c1e5-4bbd-80b5-6efb1965aeb9 for instance 35a9c645-590f-49e1-ab03-af4167f10547 on hv-cinder-84336 May 28 03:38:19.527948 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-9e93ae55-85f6-43f4-ae1c-408d0b39beab req-447927de-bee1-48b6-a56a-e1c094a56a77 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.033710 May 28 03:38:19.529406 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 168/350] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:19 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:19.545155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a9a3445f-4ac7-4a16-8d5e-99a0714d25a9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:19.546142 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a9a3445f-4ac7-4a16-8d5e-99a0714d25a9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:19.555788 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a9a3445f-4ac7-4a16-8d5e-99a0714d25a9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.556465 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a9a3445f-4ac7-4a16-8d5e-99a0714d25a9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.557482 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a9a3445f-4ac7-4a16-8d5e-99a0714d25a9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.558694 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a9a3445f-4ac7-4a16-8d5e-99a0714d25a9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.646235 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d4ff1c64-f430-46c3-932f-f45b4e2ca682 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:19.654545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d4ff1c64-f430-46c3-932f-f45b4e2ca682 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.178829 May 28 03:38:19.655125 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 183/351] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:19 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:19.660528 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-65a31913-086d-43c6-8344-1571b9914b58 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:19.661165 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-65a31913-086d-43c6-8344-1571b9914b58 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:19.667391 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-65a31913-086d-43c6-8344-1571b9914b58 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.667839 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-65a31913-086d-43c6-8344-1571b9914b58 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.668623 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-65a31913-086d-43c6-8344-1571b9914b58 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.669073 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-65a31913-086d-43c6-8344-1571b9914b58 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.743103 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a9a3445f-4ac7-4a16-8d5e-99a0714d25a9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:19.752290 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a9a3445f-4ac7-4a16-8d5e-99a0714d25a9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1700 microversion: 2.1 time: 0.214241 May 28 03:38:19.753067 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 169/352] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:19 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1700 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:19.772209 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4ca5a070-0ecd-4b62-8edd-649144f27691 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:19.779048 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4ca5a070-0ecd-4b62-8edd-649144f27691 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.779529 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4ca5a070-0ecd-4b62-8edd-649144f27691 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.803646 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4ca5a070-0ecd-4b62-8edd-649144f27691 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Going to try to start instance {{(pid=124605) start /opt/stack/nova/nova/compute/api.py:2730}} May 28 03:38:19.807453 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-65a31913-086d-43c6-8344-1571b9914b58 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:19.815777 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-65a31913-086d-43c6-8344-1571b9914b58 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1503 microversion: 2.1 time: 0.159268 May 28 03:38:19.817508 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 184/353] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:19 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1503 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:19.854228 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-68c27b33-e06e-4e97-927b-92173e745d62 req-ebd77e8d-e910-4a25-bfb4-8e40f55080f6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690", "tag": "ea42d3d0-51a0-41a4-b165-b66f57c57a62"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:19.871352 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-68c27b33-e06e-4e97-927b-92173e745d62 req-ebd77e8d-e910-4a25-bfb4-8e40f55080f6 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.876777 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-68c27b33-e06e-4e97-927b-92173e745d62 req-ebd77e8d-e910-4a25-bfb4-8e40f55080f6 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.884554 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-68c27b33-e06e-4e97-927b-92173e745d62 req-ebd77e8d-e910-4a25-bfb4-8e40f55080f6 service nova] Creating event network-changed:ea42d3d0-51a0-41a4-b165-b66f57c57a62 for instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 on hv-cinder-84336 May 28 03:38:19.889503 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-68c27b33-e06e-4e97-927b-92173e745d62 req-ebd77e8d-e910-4a25-bfb4-8e40f55080f6 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039367 May 28 03:38:19.890185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 185/354] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:38:19.950370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4ca5a070-0ecd-4b62-8edd-649144f27691 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "POST /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/action" status: 202 len: 0 microversion: 2.1 time: 0.184249 May 28 03:38:19.950975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 170/355] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:38:19 2022] POST /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/action => generated 0 bytes in 187 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 28 03:38:19.959529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3ed611c4-87c6-4271-a937-ab5b1ab8f4e5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:19.960358 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3ed611c4-87c6-4271-a937-ab5b1ab8f4e5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:19.967253 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ed611c4-87c6-4271-a937-ab5b1ab8f4e5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.967706 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ed611c4-87c6-4271-a937-ab5b1ab8f4e5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:19.968490 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ed611c4-87c6-4271-a937-ab5b1ab8f4e5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:19.968933 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ed611c4-87c6-4271-a937-ab5b1ab8f4e5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:20.103656 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3ed611c4-87c6-4271-a937-ab5b1ab8f4e5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:20.111410 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3ed611c4-87c6-4271-a937-ab5b1ab8f4e5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1709 microversion: 2.1 time: 0.156214 May 28 03:38:20.111968 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 186/356] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:19 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1709 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:20.262977 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5f1de329-33e7-492a-840d-60f398c7e8a2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:20.263838 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5f1de329-33e7-492a-840d-60f398c7e8a2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:20.271446 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5f1de329-33e7-492a-840d-60f398c7e8a2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:20.272100 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5f1de329-33e7-492a-840d-60f398c7e8a2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:20.273015 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5f1de329-33e7-492a-840d-60f398c7e8a2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:20.273634 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5f1de329-33e7-492a-840d-60f398c7e8a2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:20.409667 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5f1de329-33e7-492a-840d-60f398c7e8a2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:20.418258 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5f1de329-33e7-492a-840d-60f398c7e8a2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:20.419735 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5f1de329-33e7-492a-840d-60f398c7e8a2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:20.420871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5f1de329-33e7-492a-840d-60f398c7e8a2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1723 microversion: 2.20 time: 0.162206 May 28 03:38:20.421455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 171/357] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:20 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:20.434596 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0e849767-f4b4-403e-b9cc-5af1d10f2ded tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:20.435296 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0e849767-f4b4-403e-b9cc-5af1d10f2ded tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:20.442602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0e849767-f4b4-403e-b9cc-5af1d10f2ded tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:20.443185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0e849767-f4b4-403e-b9cc-5af1d10f2ded tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:20.444066 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0e849767-f4b4-403e-b9cc-5af1d10f2ded tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:20.444554 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0e849767-f4b4-403e-b9cc-5af1d10f2ded tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:20.475564 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0169c4a1-d2dd-4212-83ef-c0926876e5ca tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:20.476176 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0169c4a1-d2dd-4212-83ef-c0926876e5ca tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:20.484955 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0169c4a1-d2dd-4212-83ef-c0926876e5ca tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:20.485431 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0169c4a1-d2dd-4212-83ef-c0926876e5ca tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:20.486205 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0169c4a1-d2dd-4212-83ef-c0926876e5ca tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:20.486841 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0169c4a1-d2dd-4212-83ef-c0926876e5ca tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:20.551247 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0169c4a1-d2dd-4212-83ef-c0926876e5ca tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:20.558801 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0169c4a1-d2dd-4212-83ef-c0926876e5ca tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1223 microversion: 2.1 time: 0.089863 May 28 03:38:20.559367 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 172/358] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:20 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1223 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:20.582953 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0e849767-f4b4-403e-b9cc-5af1d10f2ded tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:20.593557 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0e849767-f4b4-403e-b9cc-5af1d10f2ded tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.162695 May 28 03:38:20.595400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 187/359] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:20 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:20.671756 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5e3ef62b-98ab-408a-870c-8f37e878121c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:20.672267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5e3ef62b-98ab-408a-870c-8f37e878121c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:20.678973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e3ef62b-98ab-408a-870c-8f37e878121c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:20.679423 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e3ef62b-98ab-408a-870c-8f37e878121c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:20.680212 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e3ef62b-98ab-408a-870c-8f37e878121c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:20.680651 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e3ef62b-98ab-408a-870c-8f37e878121c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:20.814047 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5e3ef62b-98ab-408a-870c-8f37e878121c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:20.821458 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5e3ef62b-98ab-408a-870c-8f37e878121c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.153787 May 28 03:38:20.822051 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 173/360] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:20 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:20.833595 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-831613eb-daa6-4bf5-bd79-cedc8b6d09d1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:20.839136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-831613eb-daa6-4bf5-bd79-cedc8b6d09d1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:20.840398 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-831613eb-daa6-4bf5-bd79-cedc8b6d09d1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:20.840844 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-831613eb-daa6-4bf5-bd79-cedc8b6d09d1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:20.841675 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-831613eb-daa6-4bf5-bd79-cedc8b6d09d1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:20.842161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-831613eb-daa6-4bf5-bd79-cedc8b6d09d1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:20.978379 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-831613eb-daa6-4bf5-bd79-cedc8b6d09d1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:20.987229 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-831613eb-daa6-4bf5-bd79-cedc8b6d09d1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1503 microversion: 2.1 time: 0.157365 May 28 03:38:20.988991 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 188/361] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:20 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1503 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:21.129757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-44f9131a-f826-44be-82e7-1015a244aa0c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:21.130821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-44f9131a-f826-44be-82e7-1015a244aa0c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:21.138609 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-44f9131a-f826-44be-82e7-1015a244aa0c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.139171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-44f9131a-f826-44be-82e7-1015a244aa0c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:21.140348 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-44f9131a-f826-44be-82e7-1015a244aa0c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.141071 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-44f9131a-f826-44be-82e7-1015a244aa0c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:21.394156 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-44f9131a-f826-44be-82e7-1015a244aa0c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:21.405716 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-44f9131a-f826-44be-82e7-1015a244aa0c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1709 microversion: 2.1 time: 0.280519 May 28 03:38:21.406305 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 174/362] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:21 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1709 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:21.439611 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b535803a-7642-4128-967c-7760f30f14e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:21.440223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b535803a-7642-4128-967c-7760f30f14e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:21.447909 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b535803a-7642-4128-967c-7760f30f14e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.448354 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b535803a-7642-4128-967c-7760f30f14e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:21.449177 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b535803a-7642-4128-967c-7760f30f14e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.449625 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b535803a-7642-4128-967c-7760f30f14e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:21.576429 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9a1ea68a-9571-4786-92a5-d589af47aa47 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:21.577218 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9a1ea68a-9571-4786-92a5-d589af47aa47 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:21.584438 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a1ea68a-9571-4786-92a5-d589af47aa47 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.585221 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a1ea68a-9571-4786-92a5-d589af47aa47 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:21.586089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a1ea68a-9571-4786-92a5-d589af47aa47 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.586597 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a1ea68a-9571-4786-92a5-d589af47aa47 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:21.727561 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9a1ea68a-9571-4786-92a5-d589af47aa47 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:21.735442 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9a1ea68a-9571-4786-92a5-d589af47aa47 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1271 microversion: 2.1 time: 0.163372 May 28 03:38:21.736055 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 175/363] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:21 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1271 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:21.750509 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9180d660-6bca-42fc-988e-14259afb9033 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:21.751110 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9180d660-6bca-42fc-988e-14259afb9033 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:21.757933 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b535803a-7642-4128-967c-7760f30f14e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:21.758164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9180d660-6bca-42fc-988e-14259afb9033 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.758283 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9180d660-6bca-42fc-988e-14259afb9033 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:21.759045 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9180d660-6bca-42fc-988e-14259afb9033 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.759615 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9180d660-6bca-42fc-988e-14259afb9033 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:21.766013 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b535803a-7642-4128-967c-7760f30f14e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:21.770233 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b535803a-7642-4128-967c-7760f30f14e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:21.770233 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b535803a-7642-4128-967c-7760f30f14e5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1666 microversion: 2.20 time: 0.333086 May 28 03:38:21.770233 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 189/364] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:21 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1666 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:21.786643 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bad662d6-d4a5-4fad-acd9-b05f5660c501 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "13c9b26d-363a-4b6b-9c6e-9f00f1b68b50"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:21.787851 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bad662d6-d4a5-4fad-acd9-b05f5660c501 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:21.794600 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bad662d6-d4a5-4fad-acd9-b05f5660c501 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.798848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bad662d6-d4a5-4fad-acd9-b05f5660c501 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:21.798848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bad662d6-d4a5-4fad-acd9-b05f5660c501 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.798848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bad662d6-d4a5-4fad-acd9-b05f5660c501 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:21.934090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9180d660-6bca-42fc-988e-14259afb9033 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:21.943120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9180d660-6bca-42fc-988e-14259afb9033 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1366 microversion: 2.1 time: 0.196761 May 28 03:38:21.943690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 176/365] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:21 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1366 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:21.948325 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-dc834cb4-0942-4669-9414-07632c659522 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:21.948923 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-dc834cb4-0942-4669-9414-07632c659522 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:21.956967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc834cb4-0942-4669-9414-07632c659522 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.957452 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc834cb4-0942-4669-9414-07632c659522 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:21.959366 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc834cb4-0942-4669-9414-07632c659522 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:21.960100 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc834cb4-0942-4669-9414-07632c659522 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:22.120871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-dc834cb4-0942-4669-9414-07632c659522 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:22.128537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-dc834cb4-0942-4669-9414-07632c659522 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.183778 May 28 03:38:22.129856 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 177/366] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:21 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:22.134353 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2101e947-cc7f-47a3-aec6-77b3e7ec441c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:22.135021 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2101e947-cc7f-47a3-aec6-77b3e7ec441c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:22.141193 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2101e947-cc7f-47a3-aec6-77b3e7ec441c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:22.142334 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2101e947-cc7f-47a3-aec6-77b3e7ec441c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:22.143142 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2101e947-cc7f-47a3-aec6-77b3e7ec441c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:22.143580 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2101e947-cc7f-47a3-aec6-77b3e7ec441c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:22.218831 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2101e947-cc7f-47a3-aec6-77b3e7ec441c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:22.232704 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2101e947-cc7f-47a3-aec6-77b3e7ec441c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1461 microversion: 2.1 time: 0.101598 May 28 03:38:22.233607 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 178/367] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:22 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1461 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:22.241517 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-be2d6622-5147-4705-8c91-bbf21d179acf req-ce979e71-cc6f-49e5-bd93-b3e0c983745d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f966dbeb-97e6-4540-acd6-e64e52e4e45b", "name": "network-vif-deleted", "tag": "ffc62fe3-a4d5-4cba-b924-8fbde57f6682"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:22.251759 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-be2d6622-5147-4705-8c91-bbf21d179acf req-ce979e71-cc6f-49e5-bd93-b3e0c983745d service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:22.252202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-be2d6622-5147-4705-8c91-bbf21d179acf req-ce979e71-cc6f-49e5-bd93-b3e0c983745d service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:22.263942 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-be2d6622-5147-4705-8c91-bbf21d179acf req-ce979e71-cc6f-49e5-bd93-b3e0c983745d service nova] Creating event network-vif-deleted:ffc62fe3-a4d5-4cba-b924-8fbde57f6682 for instance f966dbeb-97e6-4540-acd6-e64e52e4e45b on hv-cinder-84336 May 28 03:38:22.268530 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-be2d6622-5147-4705-8c91-bbf21d179acf req-ce979e71-cc6f-49e5-bd93-b3e0c983745d service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033508 May 28 03:38:22.269142 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 179/368] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:22.424326 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d834a7c1-78d7-42db-a4c4-7c625ea7dc4f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:22.424978 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d834a7c1-78d7-42db-a4c4-7c625ea7dc4f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:22.431779 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d834a7c1-78d7-42db-a4c4-7c625ea7dc4f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:22.432229 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d834a7c1-78d7-42db-a4c4-7c625ea7dc4f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:22.432997 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d834a7c1-78d7-42db-a4c4-7c625ea7dc4f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:22.433462 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d834a7c1-78d7-42db-a4c4-7c625ea7dc4f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:22.612213 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d834a7c1-78d7-42db-a4c4-7c625ea7dc4f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:22.615832 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:22.622688 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d834a7c1-78d7-42db-a4c4-7c625ea7dc4f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1709 microversion: 2.1 time: 0.203145 May 28 03:38:22.623353 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 180/369] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:22 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1709 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:22.757620 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-15f067e8-2e91-4566-934b-4e9cc5f73daa tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:22.758691 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-15f067e8-2e91-4566-934b-4e9cc5f73daa tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:22.771534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-15f067e8-2e91-4566-934b-4e9cc5f73daa tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:22.772600 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-15f067e8-2e91-4566-934b-4e9cc5f73daa tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:22.775627 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-15f067e8-2e91-4566-934b-4e9cc5f73daa tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:22.776644 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-15f067e8-2e91-4566-934b-4e9cc5f73daa tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:22.892833 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-15f067e8-2e91-4566-934b-4e9cc5f73daa tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:22.900767 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-15f067e8-2e91-4566-934b-4e9cc5f73daa tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1279 microversion: 2.1 time: 0.148827 May 28 03:38:22.905168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 181/370] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:22 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1279 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:22.913570 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bad662d6-d4a5-4fad-acd9-b05f5660c501 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "POST /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.130906 May 28 03:38:22.914663 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 190/371] 10.222.0.41 () {62 vars in 1363 bytes} [Sat May 28 03:38:21 2022] POST /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 188 bytes in 1133 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:38:22.960320 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-11b937aa-2c44-4da9-a994-7414dccc2e1d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:22.960320 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-11b937aa-2c44-4da9-a994-7414dccc2e1d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:22.966920 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-11b937aa-2c44-4da9-a994-7414dccc2e1d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:22.971417 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-11b937aa-2c44-4da9-a994-7414dccc2e1d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:22.972437 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-11b937aa-2c44-4da9-a994-7414dccc2e1d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:22.972970 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-11b937aa-2c44-4da9-a994-7414dccc2e1d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.041707 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a33b732c-79bb-4db4-acea-34335182686c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:23.042275 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a33b732c-79bb-4db4-acea-34335182686c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:23.044269 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:23.049185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a33b732c-79bb-4db4-acea-34335182686c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.049637 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a33b732c-79bb-4db4-acea-34335182686c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.050496 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a33b732c-79bb-4db4-acea-34335182686c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.051502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a33b732c-79bb-4db4-acea-34335182686c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.170155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-11b937aa-2c44-4da9-a994-7414dccc2e1d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:23.179082 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-11b937aa-2c44-4da9-a994-7414dccc2e1d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1555 microversion: 2.1 time: 0.223179 May 28 03:38:23.179849 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 182/372] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:22 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1555 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:23.187904 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-436a5514-d315-4fcc-9a90-934f50366950 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:23.188684 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-436a5514-d315-4fcc-9a90-934f50366950 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:23.195857 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-436a5514-d315-4fcc-9a90-934f50366950 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.196646 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-436a5514-d315-4fcc-9a90-934f50366950 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.197483 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-436a5514-d315-4fcc-9a90-934f50366950 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.197875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-436a5514-d315-4fcc-9a90-934f50366950 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.298269 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a33b732c-79bb-4db4-acea-34335182686c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "POST /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/action" status: 202 len: 0 microversion: 2.20 time: 0.260374 May 28 03:38:23.299236 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 191/373] 10.222.0.41 () {62 vars in 1318 bytes} [Sat May 28 03:38:23 2022] POST /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/action => generated 0 bytes in 262 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) May 28 03:38:23.305655 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-77a49330-c5eb-45e5-8aa8-d3c7d257381f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:23.306383 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-77a49330-c5eb-45e5-8aa8-d3c7d257381f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:23.313364 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-77a49330-c5eb-45e5-8aa8-d3c7d257381f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.314122 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-77a49330-c5eb-45e5-8aa8-d3c7d257381f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.315145 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-77a49330-c5eb-45e5-8aa8-d3c7d257381f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.315741 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-77a49330-c5eb-45e5-8aa8-d3c7d257381f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.343643 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:23.439864 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-77a49330-c5eb-45e5-8aa8-d3c7d257381f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.139225 May 28 03:38:23.441420 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 192/374] 10.222.0.41 () {58 vars in 1274 bytes} [Sat May 28 03:38:23 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8/os-interface => generated 284 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:23.449839 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b3c55580-a308-4d64-8800-c136cb5dc865 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:23.450553 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b3c55580-a308-4d64-8800-c136cb5dc865 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:23.460903 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3c55580-a308-4d64-8800-c136cb5dc865 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.461411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3c55580-a308-4d64-8800-c136cb5dc865 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.462471 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3c55580-a308-4d64-8800-c136cb5dc865 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.465244 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3c55580-a308-4d64-8800-c136cb5dc865 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.524519 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3c55580-a308-4d64-8800-c136cb5dc865 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.525035 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3c55580-a308-4d64-8800-c136cb5dc865 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.525523 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-b3c55580-a308-4d64-8800-c136cb5dc865 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lazy-loading 'fault' on Instance uuid f966dbeb-97e6-4540-acd6-e64e52e4e45b {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:38:23.540770 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-436a5514-d315-4fcc-9a90-934f50366950 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:23.549869 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-436a5514-d315-4fcc-9a90-934f50366950 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.365639 May 28 03:38:23.551214 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 183/375] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:23 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:23.556477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e54ecb63-fc59-42e9-9c5e-c09ae8a98bd0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:23.557347 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e54ecb63-fc59-42e9-9c5e-c09ae8a98bd0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:23.559141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=98,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:23.564969 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e54ecb63-fc59-42e9-9c5e-c09ae8a98bd0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.566046 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e54ecb63-fc59-42e9-9c5e-c09ae8a98bd0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.567223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e54ecb63-fc59-42e9-9c5e-c09ae8a98bd0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.567817 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e54ecb63-fc59-42e9-9c5e-c09ae8a98bd0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.630671 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b3c55580-a308-4d64-8800-c136cb5dc865 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:23.640196 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b3c55580-a308-4d64-8800-c136cb5dc865 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 200 len: 1303 microversion: 2.1 time: 0.196168 May 28 03:38:23.641266 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 193/376] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:23 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 1303 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:23.648885 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-841b7c00-0a30-431c-adb6-0996a05ca962 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:23.649233 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-841b7c00-0a30-431c-adb6-0996a05ca962 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:23.658930 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-841b7c00-0a30-431c-adb6-0996a05ca962 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.659537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-841b7c00-0a30-431c-adb6-0996a05ca962 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.661524 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-841b7c00-0a30-431c-adb6-0996a05ca962 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.661959 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-841b7c00-0a30-431c-adb6-0996a05ca962 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.842461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=14 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:23.846571 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e54ecb63-fc59-42e9-9c5e-c09ae8a98bd0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:23.859396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e54ecb63-fc59-42e9-9c5e-c09ae8a98bd0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:23.861273 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e54ecb63-fc59-42e9-9c5e-c09ae8a98bd0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:23.864194 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e54ecb63-fc59-42e9-9c5e-c09ae8a98bd0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1752 microversion: 2.20 time: 0.311187 May 28 03:38:23.877410 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 184/377] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:23 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1752 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:23.923212 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-656dfe73-7b43-46b4-9de6-f38e6ae6477d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:23.927436 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-656dfe73-7b43-46b4-9de6-f38e6ae6477d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:23.934430 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-656dfe73-7b43-46b4-9de6-f38e6ae6477d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.934871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-656dfe73-7b43-46b4-9de6-f38e6ae6477d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.935636 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-656dfe73-7b43-46b4-9de6-f38e6ae6477d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:23.936066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-656dfe73-7b43-46b4-9de6-f38e6ae6477d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:23.945418 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-841b7c00-0a30-431c-adb6-0996a05ca962 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:23.953402 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-841b7c00-0a30-431c-adb6-0996a05ca962 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1709 microversion: 2.1 time: 0.310844 May 28 03:38:23.954177 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 194/378] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:23 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1709 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:24.177863 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-656dfe73-7b43-46b4-9de6-f38e6ae6477d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:24.185382 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-656dfe73-7b43-46b4-9de6-f38e6ae6477d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.266070 May 28 03:38:24.185948 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 185/379] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:23 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:24.567686 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-48728969-e831-4efb-988f-93f459b5b401 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:24.568526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-48728969-e831-4efb-988f-93f459b5b401 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:24.575133 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-48728969-e831-4efb-988f-93f459b5b401 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:24.575778 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-48728969-e831-4efb-988f-93f459b5b401 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:24.576687 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-48728969-e831-4efb-988f-93f459b5b401 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:24.577298 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-48728969-e831-4efb-988f-93f459b5b401 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:24.662360 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4a668470-d3e7-43a1-8639-af747715d94f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:24.663257 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4a668470-d3e7-43a1-8639-af747715d94f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: f966dbeb-97e6-4540-acd6-e64e52e4e45b] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:24.673673 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4a668470-d3e7-43a1-8639-af747715d94f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:24.675371 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4a668470-d3e7-43a1-8639-af747715d94f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:24.675371 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4a668470-d3e7-43a1-8639-af747715d94f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:24.675642 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4a668470-d3e7-43a1-8639-af747715d94f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:24.694487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-4a668470-d3e7-43a1-8639-af747715d94f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] HTTP exception thrown: Instance f966dbeb-97e6-4540-acd6-e64e52e4e45b could not be found. May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4a668470-d3e7-43a1-8639-af747715d94f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Request method failure captured: May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b HTTP/1.1 May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZkPhzF_6H5BawHVC4z92KBLmhtOs6NLbIVH_i2VfqP_grccz5L6_LWPBcQ_hROCwzz2XoVH9-xOQ2h7VEwhP8dCbQwX81Ca2nTVHl-BDBksl53XOHp4wwh-ZZkuxVSiG7QnTPwjDI5gJD72D_8LiVxZCRusRJyg_I-FGnboU67_4Yc May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: ccf50ff938ce4fdb8d24871054e0015d May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-VolumesActionsTest-242515314 May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:38:24.696219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/ccf50ff938ce4fdb8d24871054e0015d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/ccf50ff938ce4fdb8d24871054e0015d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/ccf50ff938ce4fdb8d24871054e0015d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_ccf50ff938ce4fdb8d24871054e0015d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-VolumesActionsTest-242515314 May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: ccf50ff938ce4fdb8d24871054e0015d May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-VolumesActionsTest-242515314 May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-VolumesActionsTest-242515314-project May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: db802295be064253ae59efcbfd12be8f May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-VolumesActionsTest-242515314-project May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance f966dbeb-97e6-4540-acd6-e64e52e4e45b could not be found. May 28 03:38:24.697686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': 'f966dbeb-97e6-4540-acd6-e64e52e4e45b'} May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f966dbeb-97e6-4540-acd6-e64e52e4e45b could not be found. May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:38:24.698568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f966dbeb-97e6-4540-acd6-e64e52e4e45b could not be found. May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f966dbeb-97e6-4540-acd6-e64e52e4e45b could not be found. May 28 03:38:24.700005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:38:24.701461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4a668470-d3e7-43a1-8639-af747715d94f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Returning 404 to user: Instance f966dbeb-97e6-4540-acd6-e64e52e4e45b could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:38:24.702990 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4a668470-d3e7-43a1-8639-af747715d94f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b" status: 404 len: 111 microversion: 2.1 time: 0.047154 May 28 03:38:24.707811 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 186/380] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:24 2022] GET /compute/v2.1/servers/f966dbeb-97e6-4540-acd6-e64e52e4e45b => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 28 03:38:24.821324 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-5c9a0f50-91a9-4349-9828-e25ab3847f1a req-fc669d07-df2f-4ea0-be30-808020875aef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8714c46b-f821-4c0a-b55d-1083da5eb5b8", "tag": "bce2d221-cb90-4b9d-b2ca-1b1020b1c795"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:24.834618 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-48728969-e831-4efb-988f-93f459b5b401 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:24.838262 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-5c9a0f50-91a9-4349-9828-e25ab3847f1a req-fc669d07-df2f-4ea0-be30-808020875aef service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:24.843924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-5c9a0f50-91a9-4349-9828-e25ab3847f1a req-fc669d07-df2f-4ea0-be30-808020875aef service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:24.847503 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-48728969-e831-4efb-988f-93f459b5b401 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.280999 May 28 03:38:24.848121 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 195/381] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:24 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:24.860871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-5c9a0f50-91a9-4349-9828-e25ab3847f1a req-fc669d07-df2f-4ea0-be30-808020875aef service nova] Creating event network-changed:bce2d221-cb90-4b9d-b2ca-1b1020b1c795 for instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 on hv-cinder-84336 May 28 03:38:24.867497 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-5c9a0f50-91a9-4349-9828-e25ab3847f1a req-fc669d07-df2f-4ea0-be30-808020875aef service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050106 May 28 03:38:24.875696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 187/382] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:24.890215 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6d90548a-5598-4151-a8f9-433331d5a208 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:24.890698 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1368600162", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "networks": [{"uuid": "d0e4e4f5-23db-4c8c-a345-0058f3ae5e11"}]}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:24.891251 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-6d90548a-5598-4151-a8f9-433331d5a208 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:24.899996 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6d90548a-5598-4151-a8f9-433331d5a208 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:24.907808 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6d90548a-5598-4151-a8f9-433331d5a208 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:24.915743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6d90548a-5598-4151-a8f9-433331d5a208 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:24.920051 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6d90548a-5598-4151-a8f9-433331d5a208 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:24.964713 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.network.neutron [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] validate_networks() for [('d0e4e4f5-23db-4c8c-a345-0058f3ae5e11', None, None, None, None, None)] {{(pid=124605) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:38:25.074980 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6d90548a-5598-4151-a8f9-433331d5a208 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:25.087641 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6d90548a-5598-4151-a8f9-433331d5a208 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:25.089298 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6d90548a-5598-4151-a8f9-433331d5a208 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:25.092093 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6d90548a-5598-4151-a8f9-433331d5a208 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1752 microversion: 2.20 time: 0.210952 May 28 03:38:25.096773 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 196/383] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:24 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1752 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:25.102782 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b2d1d44e-322c-4063-8745-ebed1de7f8a6 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:25.107403 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b2d1d44e-322c-4063-8745-ebed1de7f8a6 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:25.123985 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b2d1d44e-322c-4063-8745-ebed1de7f8a6 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:25.126064 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b2d1d44e-322c-4063-8745-ebed1de7f8a6 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:25.128246 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b2d1d44e-322c-4063-8745-ebed1de7f8a6 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:25.128983 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b2d1d44e-322c-4063-8745-ebed1de7f8a6 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:25.277370 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b2d1d44e-322c-4063-8745-ebed1de7f8a6 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:25.289448 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b2d1d44e-322c-4063-8745-ebed1de7f8a6 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1713 microversion: 2.1 time: 0.190997 May 28 03:38:25.290703 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 197/384] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:25 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1713 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:25.296325 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4121ad45-2940-4de6-9c71-465f97c35b3b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:25.297052 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4121ad45-2940-4de6-9c71-465f97c35b3b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:25.306367 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4121ad45-2940-4de6-9c71-465f97c35b3b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:25.306973 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4121ad45-2940-4de6-9c71-465f97c35b3b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:25.308015 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4121ad45-2940-4de6-9c71-465f97c35b3b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:25.308875 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4121ad45-2940-4de6-9c71-465f97c35b3b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:25.457926 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4121ad45-2940-4de6-9c71-465f97c35b3b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:25.467363 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4121ad45-2940-4de6-9c71-465f97c35b3b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.175468 May 28 03:38:25.468555 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 198/385] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:25 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:25.816239 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Flavor limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:38:25.816704 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Image limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:38:25.817171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Flavor pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:38:25.817781 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Image pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:38:25.818199 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:38:25.839773 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:25.840305 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:25.841194 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:25.841739 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:25.864562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c9a229eb-390a-4a82-be5f-8103ef94db90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:25.865522 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c9a229eb-390a-4a82-be5f-8103ef94db90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:25.866756 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Getting quotas for project ccf50ff938ce4fdb8d24871054e0015d. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:38:25.872557 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c9a229eb-390a-4a82-be5f-8103ef94db90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:25.873017 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c9a229eb-390a-4a82-be5f-8103ef94db90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:25.873830 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c9a229eb-390a-4a82-be5f-8103ef94db90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:25.874044 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Getting quotas for user db802295be064253ae59efcbfd12be8f and project ccf50ff938ce4fdb8d24871054e0015d. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:38:25.874420 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c9a229eb-390a-4a82-be5f-8103ef94db90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:25.892489 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Going to run 1 instances... {{(pid=124605) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:38:25.896876 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:38:26.045634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c9a229eb-390a-4a82-be5f-8103ef94db90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:26.057959 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c9a229eb-390a-4a82-be5f-8103ef94db90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.197526 May 28 03:38:26.061186 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 199/386] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:25 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:26.074695 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-88f6809f-3f63-409d-9daa-3363fc16dd08 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.188054 May 28 03:38:26.080172 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 188/387] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:38:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 1190 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 28 03:38:26.086645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a5e114bf-fe84-411e-a26f-11bb7b540668 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:26.093809 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a5e114bf-fe84-411e-a26f-11bb7b540668 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:26.111394 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-20b80d0d-432c-4853-8873-89a7028298af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:26.112128 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-20b80d0d-432c-4853-8873-89a7028298af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:26.122980 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20b80d0d-432c-4853-8873-89a7028298af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:26.124419 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20b80d0d-432c-4853-8873-89a7028298af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:26.127373 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20b80d0d-432c-4853-8873-89a7028298af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:26.127373 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20b80d0d-432c-4853-8873-89a7028298af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:26.151415 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a5e114bf-fe84-411e-a26f-11bb7b540668 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:26.159038 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a5e114bf-fe84-411e-a26f-11bb7b540668 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1214 microversion: 2.1 time: 0.076884 May 28 03:38:26.161669 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 200/388] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:26 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1214 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:26.487095 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b4a30397-7e65-42cd-9224-9be71db104be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:26.487095 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b4a30397-7e65-42cd-9224-9be71db104be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:26.487275 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-20b80d0d-432c-4853-8873-89a7028298af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:26.499004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-20b80d0d-432c-4853-8873-89a7028298af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:26.500887 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-20b80d0d-432c-4853-8873-89a7028298af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:26.501436 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b4a30397-7e65-42cd-9224-9be71db104be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:26.502234 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b4a30397-7e65-42cd-9224-9be71db104be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:26.502601 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-20b80d0d-432c-4853-8873-89a7028298af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1754 microversion: 2.20 time: 0.395627 May 28 03:38:26.503374 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 189/389] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:26 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1754 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:26.503484 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b4a30397-7e65-42cd-9224-9be71db104be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:26.504463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b4a30397-7e65-42cd-9224-9be71db104be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:26.537407 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-d1bcd00b-8fa4-4fe4-98ad-b676125f7fc7 req-2b0b90ca-55bf-4ca9-9648-cba885e68e7e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e654573a-25d3-4e44-b150-e0fdd2ef00df", "tag": "86370b2e-5f98-49df-b01e-c1a775a9facd"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:26.569989 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-d1bcd00b-8fa4-4fe4-98ad-b676125f7fc7 req-2b0b90ca-55bf-4ca9-9648-cba885e68e7e service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:26.570445 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-d1bcd00b-8fa4-4fe4-98ad-b676125f7fc7 req-2b0b90ca-55bf-4ca9-9648-cba885e68e7e service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:26.583873 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-d1bcd00b-8fa4-4fe4-98ad-b676125f7fc7 req-2b0b90ca-55bf-4ca9-9648-cba885e68e7e service nova] Creating event network-changed:86370b2e-5f98-49df-b01e-c1a775a9facd for instance e654573a-25d3-4e44-b150-e0fdd2ef00df on hv-cinder-84336 May 28 03:38:26.593016 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-d1bcd00b-8fa4-4fe4-98ad-b676125f7fc7 req-2b0b90ca-55bf-4ca9-9648-cba885e68e7e service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058245 May 28 03:38:26.593016 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 190/390] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:26.654855 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b4a30397-7e65-42cd-9224-9be71db104be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:26.663452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b4a30397-7e65-42cd-9224-9be71db104be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.183182 May 28 03:38:26.664282 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 201/391] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:26 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:27.075570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-97b109bf-5bca-4538-b742-3efb4f6af3c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:27.076414 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-97b109bf-5bca-4538-b742-3efb4f6af3c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:27.083273 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-97b109bf-5bca-4538-b742-3efb4f6af3c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:27.083716 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-97b109bf-5bca-4538-b742-3efb4f6af3c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:27.084474 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-97b109bf-5bca-4538-b742-3efb4f6af3c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:27.084901 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-97b109bf-5bca-4538-b742-3efb4f6af3c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:27.177678 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dd46ec0f-ea58-48b7-8fec-5ac1c0a6d33d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:27.178434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dd46ec0f-ea58-48b7-8fec-5ac1c0a6d33d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:27.184977 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd46ec0f-ea58-48b7-8fec-5ac1c0a6d33d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:27.185989 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd46ec0f-ea58-48b7-8fec-5ac1c0a6d33d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:27.186775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd46ec0f-ea58-48b7-8fec-5ac1c0a6d33d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:27.187215 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd46ec0f-ea58-48b7-8fec-5ac1c0a6d33d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:27.252653 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dd46ec0f-ea58-48b7-8fec-5ac1c0a6d33d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:27.266655 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-dd46ec0f-ea58-48b7-8fec-5ac1c0a6d33d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1218 microversion: 2.1 time: 0.093197 May 28 03:38:27.278317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 202/392] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:27 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1218 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:27.329936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-97b109bf-5bca-4538-b742-3efb4f6af3c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:27.339247 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-97b109bf-5bca-4538-b742-3efb4f6af3c9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.267699 May 28 03:38:27.339749 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 191/393] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:27 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:27.522663 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0a8a8dd1-dc79-4d44-8b7f-260b9925e708 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:27.523287 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0a8a8dd1-dc79-4d44-8b7f-260b9925e708 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:27.529535 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a8a8dd1-dc79-4d44-8b7f-260b9925e708 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:27.529982 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a8a8dd1-dc79-4d44-8b7f-260b9925e708 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:27.530822 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a8a8dd1-dc79-4d44-8b7f-260b9925e708 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:27.531263 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a8a8dd1-dc79-4d44-8b7f-260b9925e708 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:27.665513 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0a8a8dd1-dc79-4d44-8b7f-260b9925e708 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:27.674056 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0a8a8dd1-dc79-4d44-8b7f-260b9925e708 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:27.675683 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0a8a8dd1-dc79-4d44-8b7f-260b9925e708 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:27.676846 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0a8a8dd1-dc79-4d44-8b7f-260b9925e708 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.158854 May 28 03:38:27.677429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 203/394] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:27 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:27.679668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8c6b3f1e-6b34-408f-8cb6-b2569c02e519 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:27.680281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8c6b3f1e-6b34-408f-8cb6-b2569c02e519 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:27.686337 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c6b3f1e-6b34-408f-8cb6-b2569c02e519 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:27.686819 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c6b3f1e-6b34-408f-8cb6-b2569c02e519 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:27.687652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c6b3f1e-6b34-408f-8cb6-b2569c02e519 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:27.688024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c6b3f1e-6b34-408f-8cb6-b2569c02e519 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:27.824979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8c6b3f1e-6b34-408f-8cb6-b2569c02e519 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:27.832671 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8c6b3f1e-6b34-408f-8cb6-b2569c02e519 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.156772 May 28 03:38:27.833451 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 192/395] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:27 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:28.284065 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-24d8b3be-7d52-4e7e-8f29-b03dea8e273d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:28.284677 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-24d8b3be-7d52-4e7e-8f29-b03dea8e273d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:28.287111 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=4 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:28.292403 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24d8b3be-7d52-4e7e-8f29-b03dea8e273d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:28.292877 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24d8b3be-7d52-4e7e-8f29-b03dea8e273d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:28.293676 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24d8b3be-7d52-4e7e-8f29-b03dea8e273d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:28.294106 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24d8b3be-7d52-4e7e-8f29-b03dea8e273d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:28.356068 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7e9b942f-afb6-4c50-a8fd-e20632c7152e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:28.356789 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7e9b942f-afb6-4c50-a8fd-e20632c7152e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:28.363587 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7e9b942f-afb6-4c50-a8fd-e20632c7152e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:28.364386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7e9b942f-afb6-4c50-a8fd-e20632c7152e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:28.365240 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7e9b942f-afb6-4c50-a8fd-e20632c7152e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:28.365737 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7e9b942f-afb6-4c50-a8fd-e20632c7152e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:28.372757 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-24d8b3be-7d52-4e7e-8f29-b03dea8e273d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:28.382933 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-24d8b3be-7d52-4e7e-8f29-b03dea8e273d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1210 microversion: 2.1 time: 0.102687 May 28 03:38:28.383839 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 204/396] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:28 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1210 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:28.493260 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7e9b942f-afb6-4c50-a8fd-e20632c7152e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:28.500799 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7e9b942f-afb6-4c50-a8fd-e20632c7152e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.148574 May 28 03:38:28.501404 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 193/397] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:28 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:28.695761 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-046a0d33-4c04-4019-add4-3f3b19a4477e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:28.696304 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-046a0d33-4c04-4019-add4-3f3b19a4477e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:28.702295 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-046a0d33-4c04-4019-add4-3f3b19a4477e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:28.702734 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-046a0d33-4c04-4019-add4-3f3b19a4477e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:28.703490 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-046a0d33-4c04-4019-add4-3f3b19a4477e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:28.703923 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-046a0d33-4c04-4019-add4-3f3b19a4477e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:28.836577 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-046a0d33-4c04-4019-add4-3f3b19a4477e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:28.846598 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-046a0d33-4c04-4019-add4-3f3b19a4477e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:28.848424 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-046a0d33-4c04-4019-add4-3f3b19a4477e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:28.849665 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-046a0d33-4c04-4019-add4-3f3b19a4477e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.158526 May 28 03:38:28.849840 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e4ab28c7-6d5e-49b1-9c6a-68be2395eb1a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:28.852516 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 205/398] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:28 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:28.852632 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e4ab28c7-6d5e-49b1-9c6a-68be2395eb1a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:28.856904 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4ab28c7-6d5e-49b1-9c6a-68be2395eb1a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:28.857450 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4ab28c7-6d5e-49b1-9c6a-68be2395eb1a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:28.858787 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4ab28c7-6d5e-49b1-9c6a-68be2395eb1a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:28.858787 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4ab28c7-6d5e-49b1-9c6a-68be2395eb1a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:29.034796 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e4ab28c7-6d5e-49b1-9c6a-68be2395eb1a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:29.039647 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e4ab28c7-6d5e-49b1-9c6a-68be2395eb1a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.193793 May 28 03:38:29.040215 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 194/399] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:28 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:29.400212 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-444ef4a3-55a1-4555-9fc3-8749d7696876 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:29.400819 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-444ef4a3-55a1-4555-9fc3-8749d7696876 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:29.407465 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-444ef4a3-55a1-4555-9fc3-8749d7696876 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:29.408107 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-444ef4a3-55a1-4555-9fc3-8749d7696876 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:29.409174 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-444ef4a3-55a1-4555-9fc3-8749d7696876 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:29.409788 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-444ef4a3-55a1-4555-9fc3-8749d7696876 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:29.476531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-444ef4a3-55a1-4555-9fc3-8749d7696876 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:29.486348 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-444ef4a3-55a1-4555-9fc3-8749d7696876 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1274 microversion: 2.1 time: 0.089842 May 28 03:38:29.498003 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 206/400] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:29 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1274 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:29.499971 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-2a05e234-34e6-4410-a603-06c415e29d0a req-b6ad106a-f718-47b3-a058-c4afd6815215 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690", "name": "network-vif-plugged", "status": "completed", "tag": "ea42d3d0-51a0-41a4-b165-b66f57c57a62"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:29.516397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-2a05e234-34e6-4410-a603-06c415e29d0a req-b6ad106a-f718-47b3-a058-c4afd6815215 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:29.516921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-2a05e234-34e6-4410-a603-06c415e29d0a req-b6ad106a-f718-47b3-a058-c4afd6815215 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:29.518957 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f4c38806-93c1-428d-b732-e8aae4888029 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:29.519728 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f4c38806-93c1-428d-b732-e8aae4888029 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:29.527105 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f4c38806-93c1-428d-b732-e8aae4888029 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:29.527621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f4c38806-93c1-428d-b732-e8aae4888029 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:29.528681 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f4c38806-93c1-428d-b732-e8aae4888029 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:29.528911 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f4c38806-93c1-428d-b732-e8aae4888029 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:29.533638 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-2a05e234-34e6-4410-a603-06c415e29d0a req-b6ad106a-f718-47b3-a058-c4afd6815215 service nova] Creating event network-vif-plugged:ea42d3d0-51a0-41a4-b165-b66f57c57a62 for instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 on hv-cinder-84336 May 28 03:38:29.538786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-2a05e234-34e6-4410-a603-06c415e29d0a req-b6ad106a-f718-47b3-a058-c4afd6815215 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042720 May 28 03:38:29.540163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 195/401] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:29.686144 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f4c38806-93c1-428d-b732-e8aae4888029 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:29.695361 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f4c38806-93c1-428d-b732-e8aae4888029 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.181067 May 28 03:38:29.695932 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 207/402] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:29 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:29.868798 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-455bbeb7-6b41-40c1-ab19-b6f0a8331611 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:29.869517 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-455bbeb7-6b41-40c1-ab19-b6f0a8331611 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:29.879515 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-455bbeb7-6b41-40c1-ab19-b6f0a8331611 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:29.881581 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-455bbeb7-6b41-40c1-ab19-b6f0a8331611 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:29.881581 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-455bbeb7-6b41-40c1-ab19-b6f0a8331611 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:29.881581 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-455bbeb7-6b41-40c1-ab19-b6f0a8331611 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:30.056152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-08b81084-cebf-4d2d-9b8e-bd04fe451d5a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:30.056767 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-08b81084-cebf-4d2d-9b8e-bd04fe451d5a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:30.062902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-08b81084-cebf-4d2d-9b8e-bd04fe451d5a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:30.063355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-08b81084-cebf-4d2d-9b8e-bd04fe451d5a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:30.064351 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-08b81084-cebf-4d2d-9b8e-bd04fe451d5a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:30.064958 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-08b81084-cebf-4d2d-9b8e-bd04fe451d5a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:30.170520 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-455bbeb7-6b41-40c1-ab19-b6f0a8331611 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:30.181321 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-455bbeb7-6b41-40c1-ab19-b6f0a8331611 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:30.184736 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-455bbeb7-6b41-40c1-ab19-b6f0a8331611 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:30.186462 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-455bbeb7-6b41-40c1-ab19-b6f0a8331611 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.322448 May 28 03:38:30.191313 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 196/403] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:29 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:30.221682 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-08b81084-cebf-4d2d-9b8e-bd04fe451d5a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:30.229878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-08b81084-cebf-4d2d-9b8e-bd04fe451d5a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.177489 May 28 03:38:30.230466 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 208/404] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:30 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:30.503719 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-502bf05b-9f83-4ca3-a147-b699d4bc3faa tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:30.504428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-502bf05b-9f83-4ca3-a147-b699d4bc3faa tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:30.511457 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-502bf05b-9f83-4ca3-a147-b699d4bc3faa tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:30.515183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-502bf05b-9f83-4ca3-a147-b699d4bc3faa tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:30.515183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-502bf05b-9f83-4ca3-a147-b699d4bc3faa tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:30.515183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-502bf05b-9f83-4ca3-a147-b699d4bc3faa tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:30.714140 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-126e2050-d4ae-4d87-90cd-234399fd3419 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:30.715757 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-126e2050-d4ae-4d87-90cd-234399fd3419 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:30.722802 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-126e2050-d4ae-4d87-90cd-234399fd3419 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:30.723629 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-126e2050-d4ae-4d87-90cd-234399fd3419 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:30.724644 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-126e2050-d4ae-4d87-90cd-234399fd3419 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:30.725331 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-126e2050-d4ae-4d87-90cd-234399fd3419 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:30.735867 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-502bf05b-9f83-4ca3-a147-b699d4bc3faa tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:30.744425 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-502bf05b-9f83-4ca3-a147-b699d4bc3faa tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.244527 May 28 03:38:30.752411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 197/405] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:30 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:30.924871 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-126e2050-d4ae-4d87-90cd-234399fd3419 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:30.933954 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-126e2050-d4ae-4d87-90cd-234399fd3419 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.224584 May 28 03:38:31.108885 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 209/406] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:30 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:31.208221 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5de9526f-bb84-4f50-8806-b44f56b9a91e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:31.209341 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5de9526f-bb84-4f50-8806-b44f56b9a91e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:31.216947 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5de9526f-bb84-4f50-8806-b44f56b9a91e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:31.217577 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5de9526f-bb84-4f50-8806-b44f56b9a91e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:31.218777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5de9526f-bb84-4f50-8806-b44f56b9a91e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:31.219420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5de9526f-bb84-4f50-8806-b44f56b9a91e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:31.246579 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d5b6e6b4-ed47-4995-ac95-afba4e132895 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:31.247658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d5b6e6b4-ed47-4995-ac95-afba4e132895 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:31.254038 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5b6e6b4-ed47-4995-ac95-afba4e132895 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:31.259471 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5b6e6b4-ed47-4995-ac95-afba4e132895 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:31.259471 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5b6e6b4-ed47-4995-ac95-afba4e132895 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:31.259471 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5b6e6b4-ed47-4995-ac95-afba4e132895 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:31.261051 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:31.456044 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5de9526f-bb84-4f50-8806-b44f56b9a91e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:31.470321 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5de9526f-bb84-4f50-8806-b44f56b9a91e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:31.470321 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5de9526f-bb84-4f50-8806-b44f56b9a91e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:31.471426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5de9526f-bb84-4f50-8806-b44f56b9a91e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.268439 May 28 03:38:31.478624 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 198/407] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:31 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:31.498945 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d5b6e6b4-ed47-4995-ac95-afba4e132895 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:31.509889 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d5b6e6b4-ed47-4995-ac95-afba4e132895 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.264830 May 28 03:38:31.510895 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 210/408] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:31 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:31.774706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a46a5b33-b0f7-4972-a634-271afa13feb3 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:31.776663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a46a5b33-b0f7-4972-a634-271afa13feb3 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:31.787271 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a46a5b33-b0f7-4972-a634-271afa13feb3 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:31.787944 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a46a5b33-b0f7-4972-a634-271afa13feb3 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:31.789135 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a46a5b33-b0f7-4972-a634-271afa13feb3 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:31.789828 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a46a5b33-b0f7-4972-a634-271afa13feb3 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:31.972052 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a46a5b33-b0f7-4972-a634-271afa13feb3 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:31.980583 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a46a5b33-b0f7-4972-a634-271afa13feb3 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.213924 May 28 03:38:31.981266 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 199/409] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:31 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:32.093259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-c0771c68-39c7-4d91-b20a-41ec36f44671 req-2830fdf5-b989-4e64-92bc-eadbff74c56c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a168e633-a260-418c-acde-848a587bd43c", "tag": "114f72ce-4108-4448-8ad0-d5f0db0c3ddf"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:32.098848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-c0771c68-39c7-4d91-b20a-41ec36f44671 req-2830fdf5-b989-4e64-92bc-eadbff74c56c service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:32.099368 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-c0771c68-39c7-4d91-b20a-41ec36f44671 req-2830fdf5-b989-4e64-92bc-eadbff74c56c service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:32.118563 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-c0771c68-39c7-4d91-b20a-41ec36f44671 req-2830fdf5-b989-4e64-92bc-eadbff74c56c service nova] Creating event network-changed:114f72ce-4108-4448-8ad0-d5f0db0c3ddf for instance a168e633-a260-418c-acde-848a587bd43c on hv-cinder-84336 May 28 03:38:32.119989 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e853ab77-b856-4594-8f9f-b46d162a41fd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:32.120617 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e853ab77-b856-4594-8f9f-b46d162a41fd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:32.130673 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e853ab77-b856-4594-8f9f-b46d162a41fd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:32.133123 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e853ab77-b856-4594-8f9f-b46d162a41fd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:32.133252 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-c0771c68-39c7-4d91-b20a-41ec36f44671 req-2830fdf5-b989-4e64-92bc-eadbff74c56c service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041469 May 28 03:38:32.134205 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 211/410] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:32.135008 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e853ab77-b856-4594-8f9f-b46d162a41fd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:32.135643 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e853ab77-b856-4594-8f9f-b46d162a41fd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:32.301492 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e853ab77-b856-4594-8f9f-b46d162a41fd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:32.304527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=100 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:32.310562 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e853ab77-b856-4594-8f9f-b46d162a41fd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.194460 May 28 03:38:32.314299 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 200/411] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:32 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:32.502017 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-70cb0c2f-3411-4127-b061-3087d6c4c4c9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:32.502716 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-70cb0c2f-3411-4127-b061-3087d6c4c4c9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:32.509639 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70cb0c2f-3411-4127-b061-3087d6c4c4c9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:32.510337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70cb0c2f-3411-4127-b061-3087d6c4c4c9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:32.511287 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70cb0c2f-3411-4127-b061-3087d6c4c4c9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:32.511887 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70cb0c2f-3411-4127-b061-3087d6c4c4c9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:32.528867 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-04754c93-a057-45f6-ba1e-f99bed002440 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:32.529867 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-04754c93-a057-45f6-ba1e-f99bed002440 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:32.536564 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-04754c93-a057-45f6-ba1e-f99bed002440 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:32.537026 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-04754c93-a057-45f6-ba1e-f99bed002440 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:32.537879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-04754c93-a057-45f6-ba1e-f99bed002440 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:32.538385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-04754c93-a057-45f6-ba1e-f99bed002440 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:32.672664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-70cb0c2f-3411-4127-b061-3087d6c4c4c9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:32.682344 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-70cb0c2f-3411-4127-b061-3087d6c4c4c9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:32.684331 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-70cb0c2f-3411-4127-b061-3087d6c4c4c9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:32.685546 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-70cb0c2f-3411-4127-b061-3087d6c4c4c9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.187779 May 28 03:38:32.696783 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 212/412] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:32 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:32.697919 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-04754c93-a057-45f6-ba1e-f99bed002440 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:32.710409 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-04754c93-a057-45f6-ba1e-f99bed002440 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.186465 May 28 03:38:32.711007 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 201/413] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:32 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:32.997624 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1d5e9d31-6654-484e-a11c-44b563dc9812 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:32.998230 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1d5e9d31-6654-484e-a11c-44b563dc9812 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:33.004789 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d5e9d31-6654-484e-a11c-44b563dc9812 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:33.005270 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d5e9d31-6654-484e-a11c-44b563dc9812 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:33.006031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d5e9d31-6654-484e-a11c-44b563dc9812 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:33.006467 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d5e9d31-6654-484e-a11c-44b563dc9812 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:33.149303 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1d5e9d31-6654-484e-a11c-44b563dc9812 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:33.149882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1d5e9d31-6654-484e-a11c-44b563dc9812 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.155608 May 28 03:38:33.158708 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 213/414] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:32 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:33.328056 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-05e44833-f0fe-4908-964f-06620d737016 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:33.328830 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-05e44833-f0fe-4908-964f-06620d737016 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:33.336839 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-05e44833-f0fe-4908-964f-06620d737016 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:33.337449 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-05e44833-f0fe-4908-964f-06620d737016 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:33.338701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-05e44833-f0fe-4908-964f-06620d737016 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:33.339375 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-05e44833-f0fe-4908-964f-06620d737016 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:33.487757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-05e44833-f0fe-4908-964f-06620d737016 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:33.500772 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-05e44833-f0fe-4908-964f-06620d737016 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.176616 May 28 03:38:33.511342 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 202/415] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:33 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:33.575698 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-57687643-2dce-4b0b-b406-e63dc180554d req-0f36d0bc-7ad5-43f8-8994-99b2959d2497 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e654573a-25d3-4e44-b150-e0fdd2ef00df", "name": "network-vif-plugged", "status": "completed", "tag": "86370b2e-5f98-49df-b01e-c1a775a9facd"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:33.584435 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-57687643-2dce-4b0b-b406-e63dc180554d req-0f36d0bc-7ad5-43f8-8994-99b2959d2497 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:33.584989 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-57687643-2dce-4b0b-b406-e63dc180554d req-0f36d0bc-7ad5-43f8-8994-99b2959d2497 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:33.599609 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-57687643-2dce-4b0b-b406-e63dc180554d req-0f36d0bc-7ad5-43f8-8994-99b2959d2497 service nova] Creating event network-vif-plugged:86370b2e-5f98-49df-b01e-c1a775a9facd for instance e654573a-25d3-4e44-b150-e0fdd2ef00df on hv-cinder-84336 May 28 03:38:33.605061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-57687643-2dce-4b0b-b406-e63dc180554d req-0f36d0bc-7ad5-43f8-8994-99b2959d2497 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033752 May 28 03:38:33.609191 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 214/416] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:33.704275 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ff32bf24-7fd5-49fe-bb7d-de9de73de90d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:33.704882 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ff32bf24-7fd5-49fe-bb7d-de9de73de90d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:33.721156 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ff32bf24-7fd5-49fe-bb7d-de9de73de90d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:33.721156 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ff32bf24-7fd5-49fe-bb7d-de9de73de90d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:33.721582 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ff32bf24-7fd5-49fe-bb7d-de9de73de90d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:33.721834 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ff32bf24-7fd5-49fe-bb7d-de9de73de90d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:33.729001 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-12467f1d-0813-4a6f-b256-3b218a854753 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:33.729897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-12467f1d-0813-4a6f-b256-3b218a854753 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:33.740933 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-12467f1d-0813-4a6f-b256-3b218a854753 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:33.741406 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-12467f1d-0813-4a6f-b256-3b218a854753 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:33.742197 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-12467f1d-0813-4a6f-b256-3b218a854753 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:33.742634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-12467f1d-0813-4a6f-b256-3b218a854753 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:33.911250 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ff32bf24-7fd5-49fe-bb7d-de9de73de90d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:33.923505 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ff32bf24-7fd5-49fe-bb7d-de9de73de90d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:33.925142 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ff32bf24-7fd5-49fe-bb7d-de9de73de90d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:33.926373 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ff32bf24-7fd5-49fe-bb7d-de9de73de90d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.226055 May 28 03:38:33.926518 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-12467f1d-0813-4a6f-b256-3b218a854753 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:33.926923 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 203/417] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:33 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:33.935895 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-12467f1d-0813-4a6f-b256-3b218a854753 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.211150 May 28 03:38:33.936893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 215/418] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:33 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:34.167068 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a95a0260-66d3-498d-a7e0-e2762b2ebdba tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:34.167886 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a95a0260-66d3-498d-a7e0-e2762b2ebdba tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:34.174047 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a95a0260-66d3-498d-a7e0-e2762b2ebdba tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:34.174486 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a95a0260-66d3-498d-a7e0-e2762b2ebdba tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:34.175238 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a95a0260-66d3-498d-a7e0-e2762b2ebdba tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:34.175663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a95a0260-66d3-498d-a7e0-e2762b2ebdba tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:34.319374 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a95a0260-66d3-498d-a7e0-e2762b2ebdba tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:34.326757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a95a0260-66d3-498d-a7e0-e2762b2ebdba tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.163663 May 28 03:38:34.327329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 204/419] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:34 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:34.518612 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1e74ff3a-aca5-4554-ac53-2599b63f582a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:34.519388 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1e74ff3a-aca5-4554-ac53-2599b63f582a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:34.526972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1e74ff3a-aca5-4554-ac53-2599b63f582a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:34.527811 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1e74ff3a-aca5-4554-ac53-2599b63f582a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:34.528593 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1e74ff3a-aca5-4554-ac53-2599b63f582a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:34.529154 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1e74ff3a-aca5-4554-ac53-2599b63f582a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:34.694986 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1e74ff3a-aca5-4554-ac53-2599b63f582a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:34.702541 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1e74ff3a-aca5-4554-ac53-2599b63f582a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.187889 May 28 03:38:34.703108 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 216/420] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:34 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:34.946143 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1232dcd4-6ed3-4d6c-aba3-4c3d0b27a61f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:34.946143 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1232dcd4-6ed3-4d6c-aba3-4c3d0b27a61f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:34.955005 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-94e3487b-517b-4fba-b61b-41bf219b72a6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:34.955229 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1232dcd4-6ed3-4d6c-aba3-4c3d0b27a61f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:34.955497 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1232dcd4-6ed3-4d6c-aba3-4c3d0b27a61f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:34.956285 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1232dcd4-6ed3-4d6c-aba3-4c3d0b27a61f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:34.956554 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-94e3487b-517b-4fba-b61b-41bf219b72a6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:34.956843 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1232dcd4-6ed3-4d6c-aba3-4c3d0b27a61f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:34.964033 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-94e3487b-517b-4fba-b61b-41bf219b72a6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:34.964837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-94e3487b-517b-4fba-b61b-41bf219b72a6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:34.965952 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-94e3487b-517b-4fba-b61b-41bf219b72a6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:34.966605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-94e3487b-517b-4fba-b61b-41bf219b72a6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:35.117814 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1232dcd4-6ed3-4d6c-aba3-4c3d0b27a61f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:35.126376 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1232dcd4-6ed3-4d6c-aba3-4c3d0b27a61f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:35.127717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-94e3487b-517b-4fba-b61b-41bf219b72a6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:35.127995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1232dcd4-6ed3-4d6c-aba3-4c3d0b27a61f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:35.129341 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1232dcd4-6ed3-4d6c-aba3-4c3d0b27a61f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.189007 May 28 03:38:35.129907 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 205/421] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:34 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:35.141144 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-94e3487b-517b-4fba-b61b-41bf219b72a6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.188985 May 28 03:38:35.150642 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 217/422] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:34 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:35.349816 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1daecd80-6758-4aba-b165-ed958165c710 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:35.350976 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1daecd80-6758-4aba-b165-ed958165c710 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:35.359807 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1daecd80-6758-4aba-b165-ed958165c710 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:35.360668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1daecd80-6758-4aba-b165-ed958165c710 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:35.365033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1daecd80-6758-4aba-b165-ed958165c710 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:35.365652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1daecd80-6758-4aba-b165-ed958165c710 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:35.508611 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1daecd80-6758-4aba-b165-ed958165c710 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:35.516179 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1daecd80-6758-4aba-b165-ed958165c710 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.172920 May 28 03:38:35.516761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 206/423] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:35 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:35.719763 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ed2d829e-02d3-44a0-b2b6-f43004d772d3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:35.720378 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ed2d829e-02d3-44a0-b2b6-f43004d772d3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:35.727322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ed2d829e-02d3-44a0-b2b6-f43004d772d3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:35.727767 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ed2d829e-02d3-44a0-b2b6-f43004d772d3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:35.728531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ed2d829e-02d3-44a0-b2b6-f43004d772d3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:35.728964 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ed2d829e-02d3-44a0-b2b6-f43004d772d3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:35.866793 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ed2d829e-02d3-44a0-b2b6-f43004d772d3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:35.874630 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ed2d829e-02d3-44a0-b2b6-f43004d772d3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.158757 May 28 03:38:35.875197 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 218/424] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:35 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:36.157299 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1589296f-6951-47fc-b1af-59640e2a3083 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:36.157898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1589296f-6951-47fc-b1af-59640e2a3083 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:36.159442 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-318d1bc8-6f0d-4a9f-8609-83ab64c66a10 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:36.159766 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:36.160508 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-318d1bc8-6f0d-4a9f-8609-83ab64c66a10 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:36.160628 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=6 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:36.163787 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=18 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:36.166172 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1589296f-6951-47fc-b1af-59640e2a3083 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:36.166624 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1589296f-6951-47fc-b1af-59640e2a3083 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:36.167403 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1589296f-6951-47fc-b1af-59640e2a3083 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:36.167842 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1589296f-6951-47fc-b1af-59640e2a3083 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:36.168832 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-318d1bc8-6f0d-4a9f-8609-83ab64c66a10 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:36.169294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-318d1bc8-6f0d-4a9f-8609-83ab64c66a10 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:36.170217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-318d1bc8-6f0d-4a9f-8609-83ab64c66a10 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:36.170647 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-318d1bc8-6f0d-4a9f-8609-83ab64c66a10 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:36.342819 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-318d1bc8-6f0d-4a9f-8609-83ab64c66a10 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:36.352567 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-318d1bc8-6f0d-4a9f-8609-83ab64c66a10 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.198011 May 28 03:38:36.353341 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 219/425] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:36 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:36.533687 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-92fbced8-aab3-480b-a0c4-404fd6af9759 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:36.534294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-92fbced8-aab3-480b-a0c4-404fd6af9759 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:36.540880 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-92fbced8-aab3-480b-a0c4-404fd6af9759 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:36.541406 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-92fbced8-aab3-480b-a0c4-404fd6af9759 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:36.542110 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-92fbced8-aab3-480b-a0c4-404fd6af9759 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:36.542541 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-92fbced8-aab3-480b-a0c4-404fd6af9759 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:36.592529 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1589296f-6951-47fc-b1af-59640e2a3083 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:36.602383 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1589296f-6951-47fc-b1af-59640e2a3083 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:36.604319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1589296f-6951-47fc-b1af-59640e2a3083 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:36.605816 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1589296f-6951-47fc-b1af-59640e2a3083 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.454287 May 28 03:38:36.607136 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 207/426] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:36 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:36.698914 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-92fbced8-aab3-480b-a0c4-404fd6af9759 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:36.706237 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-92fbced8-aab3-480b-a0c4-404fd6af9759 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.176400 May 28 03:38:36.706801 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 220/427] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:36 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:36.891864 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-67f28cca-0e6b-4969-9ae8-6899b9e6e0fb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:36.892482 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-67f28cca-0e6b-4969-9ae8-6899b9e6e0fb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:36.899127 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-67f28cca-0e6b-4969-9ae8-6899b9e6e0fb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:36.900615 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-67f28cca-0e6b-4969-9ae8-6899b9e6e0fb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:36.900615 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-67f28cca-0e6b-4969-9ae8-6899b9e6e0fb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:36.900928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-67f28cca-0e6b-4969-9ae8-6899b9e6e0fb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:37.041507 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-67f28cca-0e6b-4969-9ae8-6899b9e6e0fb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:37.051343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-67f28cca-0e6b-4969-9ae8-6899b9e6e0fb tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.163096 May 28 03:38:37.061536 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 208/428] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:36 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:37.370879 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d2e9850f-31a8-4874-aa06-2d158be42015 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:37.372467 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d2e9850f-31a8-4874-aa06-2d158be42015 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:37.381184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2e9850f-31a8-4874-aa06-2d158be42015 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:37.383330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2e9850f-31a8-4874-aa06-2d158be42015 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:37.383330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2e9850f-31a8-4874-aa06-2d158be42015 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:37.383330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2e9850f-31a8-4874-aa06-2d158be42015 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:37.523653 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d2e9850f-31a8-4874-aa06-2d158be42015 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:37.532312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d2e9850f-31a8-4874-aa06-2d158be42015 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.166017 May 28 03:38:37.544115 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 221/429] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:37 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:37.626480 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a5f18ff0-bb61-4ccc-91c0-0489f85248bc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:37.627535 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a5f18ff0-bb61-4ccc-91c0-0489f85248bc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:37.634350 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a5f18ff0-bb61-4ccc-91c0-0489f85248bc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:37.635161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a5f18ff0-bb61-4ccc-91c0-0489f85248bc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:37.636218 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a5f18ff0-bb61-4ccc-91c0-0489f85248bc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:37.636894 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a5f18ff0-bb61-4ccc-91c0-0489f85248bc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:37.724280 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0b7361fc-5178-44c5-833b-85878a02f923 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:37.724897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0b7361fc-5178-44c5-833b-85878a02f923 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:37.733710 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0b7361fc-5178-44c5-833b-85878a02f923 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:37.739299 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0b7361fc-5178-44c5-833b-85878a02f923 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:37.739299 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0b7361fc-5178-44c5-833b-85878a02f923 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:37.739299 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0b7361fc-5178-44c5-833b-85878a02f923 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:37.810748 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a5f18ff0-bb61-4ccc-91c0-0489f85248bc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:37.822464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a5f18ff0-bb61-4ccc-91c0-0489f85248bc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:37.823959 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a5f18ff0-bb61-4ccc-91c0-0489f85248bc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:37.825173 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a5f18ff0-bb61-4ccc-91c0-0489f85248bc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.202723 May 28 03:38:37.825746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 209/430] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:37 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:37.931639 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0b7361fc-5178-44c5-833b-85878a02f923 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:37.939022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0b7361fc-5178-44c5-833b-85878a02f923 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.218937 May 28 03:38:37.939583 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 222/431] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:37 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:38.068386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-493f86c1-5b29-4892-b78e-63b6c9e5d40f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:38.069540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-493f86c1-5b29-4892-b78e-63b6c9e5d40f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:38.075977 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-493f86c1-5b29-4892-b78e-63b6c9e5d40f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:38.076560 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-493f86c1-5b29-4892-b78e-63b6c9e5d40f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:38.077486 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-493f86c1-5b29-4892-b78e-63b6c9e5d40f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:38.078028 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-493f86c1-5b29-4892-b78e-63b6c9e5d40f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:38.240108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-493f86c1-5b29-4892-b78e-63b6c9e5d40f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:38.247410 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-493f86c1-5b29-4892-b78e-63b6c9e5d40f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.183522 May 28 03:38:38.247974 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 210/432] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:38 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:38.550548 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3de4ca5a-d0f4-4a42-ad78-999ae43079bf tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:38.551640 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3de4ca5a-d0f4-4a42-ad78-999ae43079bf tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:38.558098 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3de4ca5a-d0f4-4a42-ad78-999ae43079bf tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:38.558543 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3de4ca5a-d0f4-4a42-ad78-999ae43079bf tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:38.559300 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3de4ca5a-d0f4-4a42-ad78-999ae43079bf tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:38.559731 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3de4ca5a-d0f4-4a42-ad78-999ae43079bf tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:38.694619 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3de4ca5a-d0f4-4a42-ad78-999ae43079bf tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:38.703750 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3de4ca5a-d0f4-4a42-ad78-999ae43079bf tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.156950 May 28 03:38:38.704694 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 223/433] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:38 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:38.844668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-61648fe6-9e4a-484b-abd1-b205f9fe4900 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:38.845335 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-61648fe6-9e4a-484b-abd1-b205f9fe4900 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:38.851754 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-61648fe6-9e4a-484b-abd1-b205f9fe4900 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:38.852200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-61648fe6-9e4a-484b-abd1-b205f9fe4900 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:38.852980 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-61648fe6-9e4a-484b-abd1-b205f9fe4900 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:38.853435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-61648fe6-9e4a-484b-abd1-b205f9fe4900 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:38.956244 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-575f8231-dfa1-4018-bfb4-10145d93494e tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:38.956843 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-575f8231-dfa1-4018-bfb4-10145d93494e tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:38.964026 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-575f8231-dfa1-4018-bfb4-10145d93494e tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:38.964467 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-575f8231-dfa1-4018-bfb4-10145d93494e tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:38.970443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-575f8231-dfa1-4018-bfb4-10145d93494e tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:38.970443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-575f8231-dfa1-4018-bfb4-10145d93494e tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:39.008994 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-61648fe6-9e4a-484b-abd1-b205f9fe4900 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:39.021685 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-61648fe6-9e4a-484b-abd1-b205f9fe4900 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:39.023776 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-61648fe6-9e4a-484b-abd1-b205f9fe4900 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:39.025243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-61648fe6-9e4a-484b-abd1-b205f9fe4900 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.184571 May 28 03:38:39.036179 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 211/434] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:38 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:38:39.110358 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-575f8231-dfa1-4018-bfb4-10145d93494e tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:39.119044 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-575f8231-dfa1-4018-bfb4-10145d93494e tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.166498 May 28 03:38:39.119796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 224/435] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:38 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:39.265238 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9551f196-f3e4-4bff-869c-ad09542a5644 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:39.265905 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9551f196-f3e4-4bff-869c-ad09542a5644 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:39.272056 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9551f196-f3e4-4bff-869c-ad09542a5644 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:39.272509 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9551f196-f3e4-4bff-869c-ad09542a5644 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:39.273313 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9551f196-f3e4-4bff-869c-ad09542a5644 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:39.273755 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9551f196-f3e4-4bff-869c-ad09542a5644 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:39.406092 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9551f196-f3e4-4bff-869c-ad09542a5644 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:39.413730 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9551f196-f3e4-4bff-869c-ad09542a5644 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.152518 May 28 03:38:39.414304 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 212/436] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:39 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:39.720910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6451dcc8-d8b3-4e29-9a5f-e1abdfb6fe1c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:39.721846 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-6451dcc8-d8b3-4e29-9a5f-e1abdfb6fe1c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:39.729337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6451dcc8-d8b3-4e29-9a5f-e1abdfb6fe1c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:39.729791 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6451dcc8-d8b3-4e29-9a5f-e1abdfb6fe1c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:39.730565 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6451dcc8-d8b3-4e29-9a5f-e1abdfb6fe1c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:39.731006 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6451dcc8-d8b3-4e29-9a5f-e1abdfb6fe1c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:39.878439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6451dcc8-d8b3-4e29-9a5f-e1abdfb6fe1c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:39.886070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6451dcc8-d8b3-4e29-9a5f-e1abdfb6fe1c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.169091 May 28 03:38:39.886636 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 225/437] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:39 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:40.043618 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6af0cc6a-3728-4758-9526-90ffdec6a57f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:40.044214 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6af0cc6a-3728-4758-9526-90ffdec6a57f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:40.050747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6af0cc6a-3728-4758-9526-90ffdec6a57f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:40.051719 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6af0cc6a-3728-4758-9526-90ffdec6a57f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:40.052489 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6af0cc6a-3728-4758-9526-90ffdec6a57f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:40.052928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6af0cc6a-3728-4758-9526-90ffdec6a57f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:40.136723 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-81520a23-8edd-4ed6-ac7f-870716cee8b1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:40.137369 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-81520a23-8edd-4ed6-ac7f-870716cee8b1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:40.143514 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-81520a23-8edd-4ed6-ac7f-870716cee8b1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:40.143960 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-81520a23-8edd-4ed6-ac7f-870716cee8b1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:40.144742 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-81520a23-8edd-4ed6-ac7f-870716cee8b1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:40.145250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-81520a23-8edd-4ed6-ac7f-870716cee8b1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:40.191009 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6af0cc6a-3728-4758-9526-90ffdec6a57f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:40.200723 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6af0cc6a-3728-4758-9526-90ffdec6a57f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:40.205900 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6af0cc6a-3728-4758-9526-90ffdec6a57f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:40.205900 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6af0cc6a-3728-4758-9526-90ffdec6a57f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.163801 May 28 03:38:40.205900 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 213/438] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:40 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:40.431931 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-32a310ab-f307-4423-878d-50ed009ab289 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:40.432584 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-32a310ab-f307-4423-878d-50ed009ab289 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:40.439801 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-32a310ab-f307-4423-878d-50ed009ab289 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:40.441236 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-32a310ab-f307-4423-878d-50ed009ab289 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:40.442464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-32a310ab-f307-4423-878d-50ed009ab289 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:40.443135 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-32a310ab-f307-4423-878d-50ed009ab289 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:40.514179 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-81520a23-8edd-4ed6-ac7f-870716cee8b1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:40.523103 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-81520a23-8edd-4ed6-ac7f-870716cee8b1 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.390216 May 28 03:38:40.527256 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 226/439] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:40 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:40.594922 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-32a310ab-f307-4423-878d-50ed009ab289 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:40.604181 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-32a310ab-f307-4423-878d-50ed009ab289 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1371 microversion: 2.1 time: 0.176744 May 28 03:38:40.605973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 214/440] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:40 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1371 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:40.904433 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-90300c42-630f-4375-bb4e-fd63ca860d5d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:40.905152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-90300c42-630f-4375-bb4e-fd63ca860d5d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:40.914778 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90300c42-630f-4375-bb4e-fd63ca860d5d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:40.915227 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90300c42-630f-4375-bb4e-fd63ca860d5d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:40.916231 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90300c42-630f-4375-bb4e-fd63ca860d5d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:40.916663 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90300c42-630f-4375-bb4e-fd63ca860d5d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:41.042234 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-90300c42-630f-4375-bb4e-fd63ca860d5d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:41.049640 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-90300c42-630f-4375-bb4e-fd63ca860d5d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.150017 May 28 03:38:41.050207 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 227/441] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:40 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:41.221183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a717214e-4cc7-4869-9344-9ce356b6af3f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:41.221740 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a717214e-4cc7-4869-9344-9ce356b6af3f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:41.228807 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a717214e-4cc7-4869-9344-9ce356b6af3f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:41.229297 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a717214e-4cc7-4869-9344-9ce356b6af3f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:41.230062 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a717214e-4cc7-4869-9344-9ce356b6af3f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:41.230501 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a717214e-4cc7-4869-9344-9ce356b6af3f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:41.357969 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a717214e-4cc7-4869-9344-9ce356b6af3f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:41.367238 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a717214e-4cc7-4869-9344-9ce356b6af3f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:41.373296 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a717214e-4cc7-4869-9344-9ce356b6af3f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:41.373296 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a717214e-4cc7-4869-9344-9ce356b6af3f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.152739 May 28 03:38:41.380413 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 215/442] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:41 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:41.539598 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-708d959b-8a0b-41bf-9632-6f136c1abf67 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:41.540226 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-708d959b-8a0b-41bf-9632-6f136c1abf67 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:41.546287 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-708d959b-8a0b-41bf-9632-6f136c1abf67 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:41.546729 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-708d959b-8a0b-41bf-9632-6f136c1abf67 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:41.547484 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-708d959b-8a0b-41bf-9632-6f136c1abf67 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:41.547918 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-708d959b-8a0b-41bf-9632-6f136c1abf67 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:41.551879 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:41.624205 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1d65013f-169f-4948-b9e4-bf93c5ebc65b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:41.624816 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1d65013f-169f-4948-b9e4-bf93c5ebc65b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:41.630890 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1d65013f-169f-4948-b9e4-bf93c5ebc65b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:41.631368 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1d65013f-169f-4948-b9e4-bf93c5ebc65b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:41.632361 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1d65013f-169f-4948-b9e4-bf93c5ebc65b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:41.632811 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1d65013f-169f-4948-b9e4-bf93c5ebc65b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:41.681265 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-708d959b-8a0b-41bf-9632-6f136c1abf67 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:41.688605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-708d959b-8a0b-41bf-9632-6f136c1abf67 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.153443 May 28 03:38:41.689183 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 228/443] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:41 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:38:41.759651 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1d65013f-169f-4948-b9e4-bf93c5ebc65b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:41.767055 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1d65013f-169f-4948-b9e4-bf93c5ebc65b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1566 microversion: 2.1 time: 0.147224 May 28 03:38:41.767611 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 216/444] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:41 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1566 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:41.783694 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9b8e3c19-984e-4ce1-86db-ef4f2bb8ce38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:41.784182 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9b8e3c19-984e-4ce1-86db-ef4f2bb8ce38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:41.790679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b8e3c19-984e-4ce1-86db-ef4f2bb8ce38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:41.791058 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b8e3c19-984e-4ce1-86db-ef4f2bb8ce38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:41.791807 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b8e3c19-984e-4ce1-86db-ef4f2bb8ce38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:41.792241 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b8e3c19-984e-4ce1-86db-ef4f2bb8ce38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:41.868093 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9b8e3c19-984e-4ce1-86db-ef4f2bb8ce38 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.088074 May 28 03:38:41.868688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 229/445] 10.222.0.41 () {58 vars in 1274 bytes} [Sat May 28 03:38:41 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-interface => generated 285 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:42.066158 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6d51ed36-e0ad-4195-9817-0559eb9778c7 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:42.067810 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6d51ed36-e0ad-4195-9817-0559eb9778c7 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:42.074083 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6d51ed36-e0ad-4195-9817-0559eb9778c7 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:42.074538 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6d51ed36-e0ad-4195-9817-0559eb9778c7 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:42.075342 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6d51ed36-e0ad-4195-9817-0559eb9778c7 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:42.075789 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6d51ed36-e0ad-4195-9817-0559eb9778c7 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:42.091693 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=100 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:42.356236 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6d51ed36-e0ad-4195-9817-0559eb9778c7 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:42.364486 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6d51ed36-e0ad-4195-9817-0559eb9778c7 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.301490 May 28 03:38:42.370458 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 217/446] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:42 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:42.388217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e227604d-6595-463a-a190-f9fa84f8899f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:42.389658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e227604d-6595-463a-a190-f9fa84f8899f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:42.395672 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e227604d-6595-463a-a190-f9fa84f8899f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:42.396111 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e227604d-6595-463a-a190-f9fa84f8899f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:42.396877 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e227604d-6595-463a-a190-f9fa84f8899f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:42.397329 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e227604d-6595-463a-a190-f9fa84f8899f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:42.526087 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e227604d-6595-463a-a190-f9fa84f8899f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:42.534479 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e227604d-6595-463a-a190-f9fa84f8899f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:42.536480 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e227604d-6595-463a-a190-f9fa84f8899f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:42.537652 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e227604d-6595-463a-a190-f9fa84f8899f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.153847 May 28 03:38:42.538212 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 230/447] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:42 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:42.704539 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1ac55097-5e6f-4237-bc75-d2dab3264b9f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:42.705183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1ac55097-5e6f-4237-bc75-d2dab3264b9f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:42.711453 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ac55097-5e6f-4237-bc75-d2dab3264b9f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:42.711904 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ac55097-5e6f-4237-bc75-d2dab3264b9f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:42.712679 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ac55097-5e6f-4237-bc75-d2dab3264b9f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:42.713132 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ac55097-5e6f-4237-bc75-d2dab3264b9f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:43.016991 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1ac55097-5e6f-4237-bc75-d2dab3264b9f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:43.025046 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1ac55097-5e6f-4237-bc75-d2dab3264b9f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.324384 May 28 03:38:43.025630 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 218/448] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:42 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:43.345371 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-2337ea4a-803d-4512-8a3e-e43c959dd96f req-2dd72cf9-a135-47bf-9e4c-7a3434da6c40 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690", "tag": "ea42d3d0-51a0-41a4-b165-b66f57c57a62"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:43.353741 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-2337ea4a-803d-4512-8a3e-e43c959dd96f req-2dd72cf9-a135-47bf-9e4c-7a3434da6c40 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:43.354266 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-2337ea4a-803d-4512-8a3e-e43c959dd96f req-2dd72cf9-a135-47bf-9e4c-7a3434da6c40 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:43.367143 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-2337ea4a-803d-4512-8a3e-e43c959dd96f req-2dd72cf9-a135-47bf-9e4c-7a3434da6c40 service nova] Creating event network-changed:ea42d3d0-51a0-41a4-b165-b66f57c57a62 for instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 on hv-cinder-84336 May 28 03:38:43.372289 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-2337ea4a-803d-4512-8a3e-e43c959dd96f req-2dd72cf9-a135-47bf-9e4c-7a3434da6c40 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030569 May 28 03:38:43.373063 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 231/449] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:38:43.382143 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-20d7d29d-7de0-43e8-9da8-df2388bdffcb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:43.382760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-20d7d29d-7de0-43e8-9da8-df2388bdffcb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:43.389424 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20d7d29d-7de0-43e8-9da8-df2388bdffcb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:43.389877 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20d7d29d-7de0-43e8-9da8-df2388bdffcb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:43.390654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20d7d29d-7de0-43e8-9da8-df2388bdffcb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:43.391093 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-20d7d29d-7de0-43e8-9da8-df2388bdffcb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:43.521238 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-20d7d29d-7de0-43e8-9da8-df2388bdffcb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:43.533745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-20d7d29d-7de0-43e8-9da8-df2388bdffcb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.155913 May 28 03:38:43.534332 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 219/450] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:43 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:43.556391 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c81e7912-6cea-4d48-99b9-e325c63cc1f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:43.557091 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c81e7912-6cea-4d48-99b9-e325c63cc1f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:43.563907 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c81e7912-6cea-4d48-99b9-e325c63cc1f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:43.564359 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c81e7912-6cea-4d48-99b9-e325c63cc1f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:43.565152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c81e7912-6cea-4d48-99b9-e325c63cc1f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:43.565597 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c81e7912-6cea-4d48-99b9-e325c63cc1f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:43.697469 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c81e7912-6cea-4d48-99b9-e325c63cc1f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:43.706185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c81e7912-6cea-4d48-99b9-e325c63cc1f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:43.707687 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c81e7912-6cea-4d48-99b9-e325c63cc1f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:43.710992 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c81e7912-6cea-4d48-99b9-e325c63cc1f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.156787 May 28 03:38:43.710992 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 232/451] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:43 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:44.042998 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5e666613-aa7c-45c0-8b07-e3015b07e96f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:44.054758 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5e666613-aa7c-45c0-8b07-e3015b07e96f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:44.054758 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e666613-aa7c-45c0-8b07-e3015b07e96f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:44.054758 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e666613-aa7c-45c0-8b07-e3015b07e96f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:44.054758 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e666613-aa7c-45c0-8b07-e3015b07e96f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:44.055202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e666613-aa7c-45c0-8b07-e3015b07e96f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:44.182991 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5e666613-aa7c-45c0-8b07-e3015b07e96f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:44.190784 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5e666613-aa7c-45c0-8b07-e3015b07e96f tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.151883 May 28 03:38:44.191424 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 220/452] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:44 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:44.550931 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a13d4a2c-483a-4b40-8519-8954ca02f75d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:44.551565 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a13d4a2c-483a-4b40-8519-8954ca02f75d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:44.557575 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a13d4a2c-483a-4b40-8519-8954ca02f75d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:44.558137 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a13d4a2c-483a-4b40-8519-8954ca02f75d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:44.558756 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a13d4a2c-483a-4b40-8519-8954ca02f75d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:44.559185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a13d4a2c-483a-4b40-8519-8954ca02f75d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:44.683667 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a13d4a2c-483a-4b40-8519-8954ca02f75d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:44.690954 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a13d4a2c-483a-4b40-8519-8954ca02f75d tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1319 microversion: 2.1 time: 0.143966 May 28 03:38:44.691546 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 233/453] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:44 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1319 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:44.728091 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8c1a76e0-f544-4f51-8fcc-8e29d3073102 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:44.728615 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8c1a76e0-f544-4f51-8fcc-8e29d3073102 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:44.735021 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c1a76e0-f544-4f51-8fcc-8e29d3073102 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:44.735459 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c1a76e0-f544-4f51-8fcc-8e29d3073102 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:44.736213 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c1a76e0-f544-4f51-8fcc-8e29d3073102 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:44.736642 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c1a76e0-f544-4f51-8fcc-8e29d3073102 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:44.860510 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8c1a76e0-f544-4f51-8fcc-8e29d3073102 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:44.869127 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8c1a76e0-f544-4f51-8fcc-8e29d3073102 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:44.870629 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8c1a76e0-f544-4f51-8fcc-8e29d3073102 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:44.871786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8c1a76e0-f544-4f51-8fcc-8e29d3073102 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.147745 May 28 03:38:44.872355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 221/454] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:44 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:45.208011 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d4c4229c-1f20-4a85-8558-97e92491e48a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:45.208616 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d4c4229c-1f20-4a85-8558-97e92491e48a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:45.215153 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4c4229c-1f20-4a85-8558-97e92491e48a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:45.215601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4c4229c-1f20-4a85-8558-97e92491e48a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:45.216353 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4c4229c-1f20-4a85-8558-97e92491e48a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:45.216782 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4c4229c-1f20-4a85-8558-97e92491e48a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:45.356380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d4c4229c-1f20-4a85-8558-97e92491e48a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:45.363935 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d4c4229c-1f20-4a85-8558-97e92491e48a tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.160232 May 28 03:38:45.364807 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 234/455] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:45 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:45.707897 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-90bd10f6-8c49-41e3-aba0-44cb24947c8a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:45.708504 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-90bd10f6-8c49-41e3-aba0-44cb24947c8a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:45.714583 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-90bd10f6-8c49-41e3-aba0-44cb24947c8a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:45.715025 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-90bd10f6-8c49-41e3-aba0-44cb24947c8a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:45.715784 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-90bd10f6-8c49-41e3-aba0-44cb24947c8a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:45.716213 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-90bd10f6-8c49-41e3-aba0-44cb24947c8a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:45.856983 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-90bd10f6-8c49-41e3-aba0-44cb24947c8a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:45.874000 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-90bd10f6-8c49-41e3-aba0-44cb24947c8a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1508 microversion: 2.1 time: 0.169893 May 28 03:38:45.875551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 222/456] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:45 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1508 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:45.891351 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a9239f4d-c6d2-418b-ba0a-7036a02b7619 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:45.892007 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a9239f4d-c6d2-418b-ba0a-7036a02b7619 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:45.899715 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a9239f4d-c6d2-418b-ba0a-7036a02b7619 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:45.899872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a9239f4d-c6d2-418b-ba0a-7036a02b7619 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:45.900569 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a9239f4d-c6d2-418b-ba0a-7036a02b7619 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:45.901224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a9239f4d-c6d2-418b-ba0a-7036a02b7619 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:46.055326 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a9239f4d-c6d2-418b-ba0a-7036a02b7619 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:46.065346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a9239f4d-c6d2-418b-ba0a-7036a02b7619 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:46.067614 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a9239f4d-c6d2-418b-ba0a-7036a02b7619 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:46.068796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a9239f4d-c6d2-418b-ba0a-7036a02b7619 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.182130 May 28 03:38:46.069426 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 235/457] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:45 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:46.381566 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-939fd62d-4af5-40f1-afa9-8a515ea04e54 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:46.382189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-939fd62d-4af5-40f1-afa9-8a515ea04e54 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:46.388581 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-939fd62d-4af5-40f1-afa9-8a515ea04e54 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:46.389020 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-939fd62d-4af5-40f1-afa9-8a515ea04e54 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:46.389896 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-939fd62d-4af5-40f1-afa9-8a515ea04e54 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:46.390330 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-939fd62d-4af5-40f1-afa9-8a515ea04e54 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:46.534162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-939fd62d-4af5-40f1-afa9-8a515ea04e54 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:46.542042 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-939fd62d-4af5-40f1-afa9-8a515ea04e54 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.164396 May 28 03:38:46.542607 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 223/458] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:46 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:47.087588 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c12ff00e-96a4-4d17-87f6-0f5d99c2137f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:47.088233 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c12ff00e-96a4-4d17-87f6-0f5d99c2137f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:47.095013 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c12ff00e-96a4-4d17-87f6-0f5d99c2137f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:47.095457 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c12ff00e-96a4-4d17-87f6-0f5d99c2137f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:47.096224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c12ff00e-96a4-4d17-87f6-0f5d99c2137f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:47.096658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c12ff00e-96a4-4d17-87f6-0f5d99c2137f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:47.223939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c12ff00e-96a4-4d17-87f6-0f5d99c2137f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:47.232660 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c12ff00e-96a4-4d17-87f6-0f5d99c2137f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:47.234169 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c12ff00e-96a4-4d17-87f6-0f5d99c2137f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:47.235324 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c12ff00e-96a4-4d17-87f6-0f5d99c2137f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.151628 May 28 03:38:47.235887 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 236/459] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:47 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:47.559243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-74614527-6232-4e81-9d90-aee1fe57a973 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:47.559844 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-74614527-6232-4e81-9d90-aee1fe57a973 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:47.566460 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-74614527-6232-4e81-9d90-aee1fe57a973 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:47.567494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-74614527-6232-4e81-9d90-aee1fe57a973 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:47.570085 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-74614527-6232-4e81-9d90-aee1fe57a973 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:47.570788 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-74614527-6232-4e81-9d90-aee1fe57a973 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:47.722939 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-74614527-6232-4e81-9d90-aee1fe57a973 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:47.730406 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-74614527-6232-4e81-9d90-aee1fe57a973 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.174987 May 28 03:38:47.730967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 224/460] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:47 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:48.253745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1c33e1d8-ab18-427b-bc7f-91347044aa5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:48.254362 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1c33e1d8-ab18-427b-bc7f-91347044aa5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:48.260829 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1c33e1d8-ab18-427b-bc7f-91347044aa5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:48.261313 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1c33e1d8-ab18-427b-bc7f-91347044aa5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:48.262072 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1c33e1d8-ab18-427b-bc7f-91347044aa5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:48.262506 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1c33e1d8-ab18-427b-bc7f-91347044aa5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:48.399921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1c33e1d8-ab18-427b-bc7f-91347044aa5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:48.411769 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1c33e1d8-ab18-427b-bc7f-91347044aa5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:48.427612 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1c33e1d8-ab18-427b-bc7f-91347044aa5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:48.427612 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1c33e1d8-ab18-427b-bc7f-91347044aa5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.164880 May 28 03:38:48.427612 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 237/461] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:48 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:48.748173 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ba6b4776-9b55-4033-a3be-c65e7461a37c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:48.748736 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ba6b4776-9b55-4033-a3be-c65e7461a37c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:48.755792 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ba6b4776-9b55-4033-a3be-c65e7461a37c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:48.756243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ba6b4776-9b55-4033-a3be-c65e7461a37c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:48.757006 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ba6b4776-9b55-4033-a3be-c65e7461a37c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:48.757479 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ba6b4776-9b55-4033-a3be-c65e7461a37c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:48.885599 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ba6b4776-9b55-4033-a3be-c65e7461a37c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:48.893613 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ba6b4776-9b55-4033-a3be-c65e7461a37c tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.149249 May 28 03:38:48.894190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 225/462] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:48 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:49.434749 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-03ba4952-51a9-43b9-82a4-32c4adf6fef9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:49.435341 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-03ba4952-51a9-43b9-82a4-32c4adf6fef9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:49.437993 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:49.442789 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-03ba4952-51a9-43b9-82a4-32c4adf6fef9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:49.443243 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-03ba4952-51a9-43b9-82a4-32c4adf6fef9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:49.444020 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-03ba4952-51a9-43b9-82a4-32c4adf6fef9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:49.444461 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-03ba4952-51a9-43b9-82a4-32c4adf6fef9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:49.570108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-81d46bb3-cecf-4eca-a2f0-bef879f10902 req-bcc34f13-9696-4356-8883-0b151ea29fe6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a168e633-a260-418c-acde-848a587bd43c", "name": "network-vif-plugged", "status": "completed", "tag": "114f72ce-4108-4448-8ad0-d5f0db0c3ddf"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:49.581526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-81d46bb3-cecf-4eca-a2f0-bef879f10902 req-bcc34f13-9696-4356-8883-0b151ea29fe6 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:49.582122 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-81d46bb3-cecf-4eca-a2f0-bef879f10902 req-bcc34f13-9696-4356-8883-0b151ea29fe6 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:49.595208 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-81d46bb3-cecf-4eca-a2f0-bef879f10902 req-bcc34f13-9696-4356-8883-0b151ea29fe6 service nova] Creating event network-vif-plugged:114f72ce-4108-4448-8ad0-d5f0db0c3ddf for instance a168e633-a260-418c-acde-848a587bd43c on hv-cinder-84336 May 28 03:38:49.601226 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-81d46bb3-cecf-4eca-a2f0-bef879f10902 req-bcc34f13-9696-4356-8883-0b151ea29fe6 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036855 May 28 03:38:49.602208 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 226/463] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:49.611300 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-03ba4952-51a9-43b9-82a4-32c4adf6fef9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:49.622747 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-03ba4952-51a9-43b9-82a4-32c4adf6fef9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:49.624614 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-03ba4952-51a9-43b9-82a4-32c4adf6fef9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:49.625826 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-03ba4952-51a9-43b9-82a4-32c4adf6fef9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.194943 May 28 03:38:49.626379 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 238/464] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:49 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:49.913225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-50e8dbf7-0df2-4c95-a4b2-a64b7238dacd tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:49.914107 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-50e8dbf7-0df2-4c95-a4b2-a64b7238dacd tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:49.916198 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:49.921548 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-50e8dbf7-0df2-4c95-a4b2-a64b7238dacd tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:49.922067 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-50e8dbf7-0df2-4c95-a4b2-a64b7238dacd tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:49.923037 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-50e8dbf7-0df2-4c95-a4b2-a64b7238dacd tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:49.923565 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-50e8dbf7-0df2-4c95-a4b2-a64b7238dacd tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:50.201709 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-50e8dbf7-0df2-4c95-a4b2-a64b7238dacd tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:50.209184 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-50e8dbf7-0df2-4c95-a4b2-a64b7238dacd tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.301710 May 28 03:38:50.212486 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 227/465] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:49 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:50.645534 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0012a17b-25c5-42d3-9b28-ad3e4f7de68c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:50.646683 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0012a17b-25c5-42d3-9b28-ad3e4f7de68c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:50.653333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0012a17b-25c5-42d3-9b28-ad3e4f7de68c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:50.653810 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0012a17b-25c5-42d3-9b28-ad3e4f7de68c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:50.654612 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0012a17b-25c5-42d3-9b28-ad3e4f7de68c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:50.655049 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0012a17b-25c5-42d3-9b28-ad3e4f7de68c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:50.784498 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0012a17b-25c5-42d3-9b28-ad3e4f7de68c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:50.792737 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0012a17b-25c5-42d3-9b28-ad3e4f7de68c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:50.794254 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0012a17b-25c5-42d3-9b28-ad3e4f7de68c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:50.795399 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0012a17b-25c5-42d3-9b28-ad3e4f7de68c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.155125 May 28 03:38:50.795956 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 239/466] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:50 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:51.225563 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-06efe1a8-5f68-4534-b2ff-51ba6e5ff962 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:51.226209 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-06efe1a8-5f68-4534-b2ff-51ba6e5ff962 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:51.232807 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06efe1a8-5f68-4534-b2ff-51ba6e5ff962 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:51.233305 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06efe1a8-5f68-4534-b2ff-51ba6e5ff962 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:51.234076 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06efe1a8-5f68-4534-b2ff-51ba6e5ff962 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:51.234557 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06efe1a8-5f68-4534-b2ff-51ba6e5ff962 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:51.372505 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-06efe1a8-5f68-4534-b2ff-51ba6e5ff962 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:51.380747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-06efe1a8-5f68-4534-b2ff-51ba6e5ff962 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1314 microversion: 2.1 time: 0.158439 May 28 03:38:51.381401 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 228/467] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:51 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:51.814356 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a945ca21-5079-468b-bcd9-39ab7c768b96 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:51.814971 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a945ca21-5079-468b-bcd9-39ab7c768b96 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:51.822194 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a945ca21-5079-468b-bcd9-39ab7c768b96 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:51.822680 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a945ca21-5079-468b-bcd9-39ab7c768b96 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:51.823485 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a945ca21-5079-468b-bcd9-39ab7c768b96 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:51.823986 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a945ca21-5079-468b-bcd9-39ab7c768b96 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:51.956457 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a945ca21-5079-468b-bcd9-39ab7c768b96 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:51.964849 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a945ca21-5079-468b-bcd9-39ab7c768b96 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:51.966377 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a945ca21-5079-468b-bcd9-39ab7c768b96 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:51.967525 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a945ca21-5079-468b-bcd9-39ab7c768b96 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.157527 May 28 03:38:51.968081 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 240/468] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:51 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:38:52.397411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7484477b-4e2d-490f-9d6b-9b12eceef687 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:52.398129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7484477b-4e2d-490f-9d6b-9b12eceef687 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:52.404547 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7484477b-4e2d-490f-9d6b-9b12eceef687 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:52.404994 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7484477b-4e2d-490f-9d6b-9b12eceef687 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:52.405858 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7484477b-4e2d-490f-9d6b-9b12eceef687 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:52.406293 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7484477b-4e2d-490f-9d6b-9b12eceef687 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:52.542627 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7484477b-4e2d-490f-9d6b-9b12eceef687 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:52.550030 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7484477b-4e2d-490f-9d6b-9b12eceef687 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1499 microversion: 2.1 time: 0.156551 May 28 03:38:52.550597 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 229/469] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:52 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1499 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:52.986259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8d65dcc3-201c-41d7-af13-42b93eb7039c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:52.986884 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8d65dcc3-201c-41d7-af13-42b93eb7039c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:52.994430 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d65dcc3-201c-41d7-af13-42b93eb7039c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:52.995216 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d65dcc3-201c-41d7-af13-42b93eb7039c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:52.996134 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d65dcc3-201c-41d7-af13-42b93eb7039c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:52.996566 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d65dcc3-201c-41d7-af13-42b93eb7039c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:53.129465 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8d65dcc3-201c-41d7-af13-42b93eb7039c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:53.137848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8d65dcc3-201c-41d7-af13-42b93eb7039c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:53.139957 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8d65dcc3-201c-41d7-af13-42b93eb7039c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:53.141200 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8d65dcc3-201c-41d7-af13-42b93eb7039c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.159038 May 28 03:38:53.147796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 241/470] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:52 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:53.511358 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-868e3dce-e52a-4f09-b2c0-82701e64583d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:53.511946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-868e3dce-e52a-4f09-b2c0-82701e64583d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:53.518832 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-868e3dce-e52a-4f09-b2c0-82701e64583d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:53.519272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-868e3dce-e52a-4f09-b2c0-82701e64583d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:53.520069 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-868e3dce-e52a-4f09-b2c0-82701e64583d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:53.520494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-868e3dce-e52a-4f09-b2c0-82701e64583d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:53.545734 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-868e3dce-e52a-4f09-b2c0-82701e64583d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:38:53.547998 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-c7c92c3d-751d-4cd8-a757-f7ee902d53af req-7f11655b-de8b-4be6-a8ee-ecb4e4b7405b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35a9c645-590f-49e1-ab03-af4167f10547", "name": "network-vif-plugged", "status": "completed", "tag": "97df7472-c1e5-4bbd-80b5-6efb1965aeb9"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:53.556806 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-c7c92c3d-751d-4cd8-a757-f7ee902d53af req-7f11655b-de8b-4be6-a8ee-ecb4e4b7405b service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:53.557276 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-c7c92c3d-751d-4cd8-a757-f7ee902d53af req-7f11655b-de8b-4be6-a8ee-ecb4e4b7405b service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:53.568412 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-c7c92c3d-751d-4cd8-a757-f7ee902d53af req-7f11655b-de8b-4be6-a8ee-ecb4e4b7405b service nova] Creating event network-vif-plugged:97df7472-c1e5-4bbd-80b5-6efb1965aeb9 for instance 35a9c645-590f-49e1-ab03-af4167f10547 on hv-cinder-84336 May 28 03:38:53.573884 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-c7c92c3d-751d-4cd8-a757-f7ee902d53af req-7f11655b-de8b-4be6-a8ee-ecb4e4b7405b service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029971 May 28 03:38:53.574482 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 242/471] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:38:53.901582 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-868e3dce-e52a-4f09-b2c0-82701e64583d tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "DELETE /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 204 len: 0 microversion: 2.1 time: 0.394032 May 28 03:38:53.902966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 230/472] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:38:53 2022] DELETE /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 0 bytes in 396 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:38:53.911470 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bd5f11e4-d666-4f21-af5d-85b60259d8f9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:53.912214 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bd5f11e4-d666-4f21-af5d-85b60259d8f9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:53.921045 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd5f11e4-d666-4f21-af5d-85b60259d8f9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:53.921466 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd5f11e4-d666-4f21-af5d-85b60259d8f9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:53.922249 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd5f11e4-d666-4f21-af5d-85b60259d8f9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:53.922682 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd5f11e4-d666-4f21-af5d-85b60259d8f9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:54.053078 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bd5f11e4-d666-4f21-af5d-85b60259d8f9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:54.060579 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bd5f11e4-d666-4f21-af5d-85b60259d8f9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1505 microversion: 2.1 time: 0.153455 May 28 03:38:54.061158 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 243/473] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:53 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1505 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:54.160486 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1594ed31-7425-4d15-9ca1-a7f711f500fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:54.161242 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1594ed31-7425-4d15-9ca1-a7f711f500fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:54.167308 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1594ed31-7425-4d15-9ca1-a7f711f500fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:54.167752 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1594ed31-7425-4d15-9ca1-a7f711f500fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:54.168507 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1594ed31-7425-4d15-9ca1-a7f711f500fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:54.168938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1594ed31-7425-4d15-9ca1-a7f711f500fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:54.300885 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1594ed31-7425-4d15-9ca1-a7f711f500fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:54.310116 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1594ed31-7425-4d15-9ca1-a7f711f500fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:54.312024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1594ed31-7425-4d15-9ca1-a7f711f500fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:54.312854 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1594ed31-7425-4d15-9ca1-a7f711f500fa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.156628 May 28 03:38:54.313486 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 231/474] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:54 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:55.077923 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-7a7dd256-5d7b-450e-8a76-46c6870a2d36 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:55.078558 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-7a7dd256-5d7b-450e-8a76-46c6870a2d36 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:55.085091 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7a7dd256-5d7b-450e-8a76-46c6870a2d36 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:55.085558 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7a7dd256-5d7b-450e-8a76-46c6870a2d36 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:55.086313 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7a7dd256-5d7b-450e-8a76-46c6870a2d36 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:55.086743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7a7dd256-5d7b-450e-8a76-46c6870a2d36 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:55.217402 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-7a7dd256-5d7b-450e-8a76-46c6870a2d36 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:55.224750 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-7a7dd256-5d7b-450e-8a76-46c6870a2d36 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1505 microversion: 2.1 time: 0.150657 May 28 03:38:55.225333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 244/475] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:55 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1505 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:55.333711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-aeefd9df-dc30-46ae-8fdf-c7cfc4f52cca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:55.334311 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-aeefd9df-dc30-46ae-8fdf-c7cfc4f52cca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:55.342149 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-aeefd9df-dc30-46ae-8fdf-c7cfc4f52cca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:55.342585 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-aeefd9df-dc30-46ae-8fdf-c7cfc4f52cca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:55.343339 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-aeefd9df-dc30-46ae-8fdf-c7cfc4f52cca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:55.343769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-aeefd9df-dc30-46ae-8fdf-c7cfc4f52cca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:55.480586 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-aeefd9df-dc30-46ae-8fdf-c7cfc4f52cca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:55.488901 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-aeefd9df-dc30-46ae-8fdf-c7cfc4f52cca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:55.491246 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-aeefd9df-dc30-46ae-8fdf-c7cfc4f52cca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:55.492385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-aeefd9df-dc30-46ae-8fdf-c7cfc4f52cca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.164459 May 28 03:38:55.492944 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 232/476] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:55 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:56.241946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-11ee359d-4e65-4f9f-805d-e2712fec6c15 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:56.242475 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-11ee359d-4e65-4f9f-805d-e2712fec6c15 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:56.248905 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-11ee359d-4e65-4f9f-805d-e2712fec6c15 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:56.249386 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-11ee359d-4e65-4f9f-805d-e2712fec6c15 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:56.250155 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-11ee359d-4e65-4f9f-805d-e2712fec6c15 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:56.250594 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-11ee359d-4e65-4f9f-805d-e2712fec6c15 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:56.392449 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-11ee359d-4e65-4f9f-805d-e2712fec6c15 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:56.399914 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-11ee359d-4e65-4f9f-805d-e2712fec6c15 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1505 microversion: 2.1 time: 0.162180 May 28 03:38:56.400485 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 245/477] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:56 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1505 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:56.511808 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-fcb48f79-f7ab-4d6c-ab87-d8e862b351fd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:56.512458 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-fcb48f79-f7ab-4d6c-ab87-d8e862b351fd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:56.519108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fcb48f79-f7ab-4d6c-ab87-d8e862b351fd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:56.519560 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fcb48f79-f7ab-4d6c-ab87-d8e862b351fd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:56.520330 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fcb48f79-f7ab-4d6c-ab87-d8e862b351fd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:56.520770 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fcb48f79-f7ab-4d6c-ab87-d8e862b351fd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:56.523476 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:56.752428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-fcb48f79-f7ab-4d6c-ab87-d8e862b351fd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:56.760046 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-fcb48f79-f7ab-4d6c-ab87-d8e862b351fd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:56.761600 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-fcb48f79-f7ab-4d6c-ab87-d8e862b351fd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:56.763228 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-fcb48f79-f7ab-4d6c-ab87-d8e862b351fd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.256106 May 28 03:38:56.763739 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 233/478] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:56 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:56.946045 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-749663e6-10c3-4bde-9ff5-b17da7818a06 req-c717c037-a0c6-45b7-ba1e-def5710e145a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a168e633-a260-418c-acde-848a587bd43c", "name": "network-vif-deleted", "tag": "114f72ce-4108-4448-8ad0-d5f0db0c3ddf"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:38:56.954241 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-749663e6-10c3-4bde-9ff5-b17da7818a06 req-c717c037-a0c6-45b7-ba1e-def5710e145a service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:56.954681 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-749663e6-10c3-4bde-9ff5-b17da7818a06 req-c717c037-a0c6-45b7-ba1e-def5710e145a service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:56.965250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-749663e6-10c3-4bde-9ff5-b17da7818a06 req-c717c037-a0c6-45b7-ba1e-def5710e145a service nova] Creating event network-vif-deleted:114f72ce-4108-4448-8ad0-d5f0db0c3ddf for instance a168e633-a260-418c-acde-848a587bd43c on hv-cinder-84336 May 28 03:38:56.970463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-749663e6-10c3-4bde-9ff5-b17da7818a06 req-c717c037-a0c6-45b7-ba1e-def5710e145a service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.027814 May 28 03:38:56.971111 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 246/479] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:38:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:38:57.417688 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0edc6cbd-623e-4483-82f6-9d56944b48e9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:57.420135 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0edc6cbd-623e-4483-82f6-9d56944b48e9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:57.431181 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0edc6cbd-623e-4483-82f6-9d56944b48e9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:57.431683 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0edc6cbd-623e-4483-82f6-9d56944b48e9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:57.432591 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0edc6cbd-623e-4483-82f6-9d56944b48e9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:57.432856 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0edc6cbd-623e-4483-82f6-9d56944b48e9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:57.463547 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0edc6cbd-623e-4483-82f6-9d56944b48e9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:57.463995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0edc6cbd-623e-4483-82f6-9d56944b48e9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:57.464448 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-0edc6cbd-623e-4483-82f6-9d56944b48e9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lazy-loading 'fault' on Instance uuid a168e633-a260-418c-acde-848a587bd43c {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:38:57.509093 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0edc6cbd-623e-4483-82f6-9d56944b48e9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db802295be064253ae59efcbfd12be8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ccf50ff938ce4fdb8d24871054e0015d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:57.516421 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0edc6cbd-623e-4483-82f6-9d56944b48e9 tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 200 len: 1304 microversion: 2.1 time: 0.102654 May 28 03:38:57.517015 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 234/480] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:57 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 1304 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:38:57.782463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-40503253-61c7-47a3-b9f8-88cd77f7c5c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:57.783248 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-40503253-61c7-47a3-b9f8-88cd77f7c5c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:57.789369 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-40503253-61c7-47a3-b9f8-88cd77f7c5c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:57.789824 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-40503253-61c7-47a3-b9f8-88cd77f7c5c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:57.790596 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-40503253-61c7-47a3-b9f8-88cd77f7c5c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:57.791036 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-40503253-61c7-47a3-b9f8-88cd77f7c5c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:57.794989 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:57.931277 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-40503253-61c7-47a3-b9f8-88cd77f7c5c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:57.942296 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-40503253-61c7-47a3-b9f8-88cd77f7c5c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:57.953580 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-40503253-61c7-47a3-b9f8-88cd77f7c5c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:57.953580 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-40503253-61c7-47a3-b9f8-88cd77f7c5c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.170915 May 28 03:38:57.953580 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 247/481] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:57 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:38:58.533963 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-00800df9-432a-49b3-b2f1-162be8d0ce1b tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:58.534576 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-00800df9-432a-49b3-b2f1-162be8d0ce1b tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] [instance: a168e633-a260-418c-acde-848a587bd43c] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:58.541696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-00800df9-432a-49b3-b2f1-162be8d0ce1b tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:58.542141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-00800df9-432a-49b3-b2f1-162be8d0ce1b tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:58.542902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-00800df9-432a-49b3-b2f1-162be8d0ce1b tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:58.543340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-00800df9-432a-49b3-b2f1-162be8d0ce1b tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:58.559313 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-00800df9-432a-49b3-b2f1-162be8d0ce1b tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] HTTP exception thrown: Instance a168e633-a260-418c-acde-848a587bd43c could not be found. May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-00800df9-432a-49b3-b2f1-162be8d0ce1b tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Request method failure captured: May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c HTTP/1.1 May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZkPhzF_6H5BawHVC4z92KBLmhtOs6NLbIVH_i2VfqP_grccz5L6_LWPBcQ_hROCwzz2XoVH9-xOQ2h7VEwhP8dCbQwX81Ca2nTVHl-BDBksl53XOHp4wwh-ZZkuxVSiG7QnTPwjDI5gJD72D_8LiVxZCRusRJyg_I-FGnboU67_4Yc May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: ccf50ff938ce4fdb8d24871054e0015d May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-VolumesActionsTest-242515314 May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:38:58.560965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/ccf50ff938ce4fdb8d24871054e0015d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/ccf50ff938ce4fdb8d24871054e0015d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/ccf50ff938ce4fdb8d24871054e0015d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_ccf50ff938ce4fdb8d24871054e0015d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-VolumesActionsTest-242515314 May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: ccf50ff938ce4fdb8d24871054e0015d May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-VolumesActionsTest-242515314 May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-VolumesActionsTest-242515314-project May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: db802295be064253ae59efcbfd12be8f May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-VolumesActionsTest-242515314-project May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance a168e633-a260-418c-acde-848a587bd43c could not be found. May 28 03:38:58.562699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': 'a168e633-a260-418c-acde-848a587bd43c'} May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a168e633-a260-418c-acde-848a587bd43c could not be found. May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:38:58.563701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a168e633-a260-418c-acde-848a587bd43c could not be found. May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a168e633-a260-418c-acde-848a587bd43c could not be found. May 28 03:38:58.565178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:38:58.566468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-00800df9-432a-49b3-b2f1-162be8d0ce1b tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] Returning 404 to user: Instance a168e633-a260-418c-acde-848a587bd43c could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:38:58.566468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-00800df9-432a-49b3-b2f1-162be8d0ce1b tempest-VolumesActionsTest-242515314 tempest-VolumesActionsTest-242515314-project] 10.222.0.41 "GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c" status: 404 len: 111 microversion: 2.1 time: 0.032525 May 28 03:38:58.567348 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 235/482] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:38:58 2022] GET /compute/v2.1/servers/a168e633-a260-418c-acde-848a587bd43c => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:38:58.967719 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0b15cbcf-731a-4d51-ac23-e80ce53ef1e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:38:58.968327 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0b15cbcf-731a-4d51-ac23-e80ce53ef1e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:38:58.975285 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0b15cbcf-731a-4d51-ac23-e80ce53ef1e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:58.976060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0b15cbcf-731a-4d51-ac23-e80ce53ef1e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:58.976850 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0b15cbcf-731a-4d51-ac23-e80ce53ef1e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:38:58.977317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0b15cbcf-731a-4d51-ac23-e80ce53ef1e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:38:59.116497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0b15cbcf-731a-4d51-ac23-e80ce53ef1e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:59.125074 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0b15cbcf-731a-4d51-ac23-e80ce53ef1e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:59.126602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0b15cbcf-731a-4d51-ac23-e80ce53ef1e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:38:59.127754 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0b15cbcf-731a-4d51-ac23-e80ce53ef1e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1810 microversion: 2.20 time: 0.163878 May 28 03:38:59.128345 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 248/483] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:38:58 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1810 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:39:00.146991 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f913cb03-1ed6-438e-a5e1-69e11be54928 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:00.147526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f913cb03-1ed6-438e-a5e1-69e11be54928 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:00.154110 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f913cb03-1ed6-438e-a5e1-69e11be54928 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:00.154550 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f913cb03-1ed6-438e-a5e1-69e11be54928 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:00.155300 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f913cb03-1ed6-438e-a5e1-69e11be54928 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:00.155729 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f913cb03-1ed6-438e-a5e1-69e11be54928 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:00.286984 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f913cb03-1ed6-438e-a5e1-69e11be54928 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:00.295340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f913cb03-1ed6-438e-a5e1-69e11be54928 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:00.296837 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f913cb03-1ed6-438e-a5e1-69e11be54928 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:00.298023 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f913cb03-1ed6-438e-a5e1-69e11be54928 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1797 microversion: 2.20 time: 0.155518 May 28 03:39:00.298578 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 236/484] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:39:00 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1797 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:39:09.004782 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-873de012-5e84-4c22-8b5a-8bf8b68926e5 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "89be344a-1f24-4f89-b3fb-aa83f3d2f392", "device": "/dev/sdb"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:09.007150 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-873de012-5e84-4c22-8b5a-8bf8b68926e5 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:09.009896 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=21 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:09.015026 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-873de012-5e84-4c22-8b5a-8bf8b68926e5 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:09.015558 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-873de012-5e84-4c22-8b5a-8bf8b68926e5 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:09.016420 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-873de012-5e84-4c22-8b5a-8bf8b68926e5 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:09.016799 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-873de012-5e84-4c22-8b5a-8bf8b68926e5 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:09.860286 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-873de012-5e84-4c22-8b5a-8bf8b68926e5 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "POST /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.859516 May 28 03:39:09.867494 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 249/485] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:39:08 2022] POST /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df/os-volume_attachments => generated 194 bytes in 867 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:39:12.421384 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:39:12.422262 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Sat May 28 03:39:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:39:22.678839 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-26c0b3c7-232e-4925-96d0-d4f7be8ada5d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:22.679400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-26c0b3c7-232e-4925-96d0-d4f7be8ada5d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:22.681588 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:22.682249 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=21 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:22.687240 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26c0b3c7-232e-4925-96d0-d4f7be8ada5d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:22.692935 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26c0b3c7-232e-4925-96d0-d4f7be8ada5d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:22.692935 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26c0b3c7-232e-4925-96d0-d4f7be8ada5d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:22.692935 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26c0b3c7-232e-4925-96d0-d4f7be8ada5d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:22.812623 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-26c0b3c7-232e-4925-96d0-d4f7be8ada5d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lazy-loading 'services' on Instance uuid da053642-eac6-4484-9557-f0b878712cf2 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:39:23.048276 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-26c0b3c7-232e-4925-96d0-d4f7be8ada5d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "DELETE /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/os-volume_attachments/181f3f5a-3875-45bf-a1fb-7c09612282f4" status: 202 len: 0 microversion: 2.1 time: 0.373292 May 28 03:39:23.048854 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 237/486] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:39:22 2022] DELETE /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/os-volume_attachments/181f3f5a-3875-45bf-a1fb-7c09612282f4 => generated 0 bytes in 375 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:39:27.313746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=2 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:27.314527 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=21,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:27.318662 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d71c2e30-4f82-40d8-810a-d80c77d1752a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:27.326070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d71c2e30-4f82-40d8-810a-d80c77d1752a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:27.326522 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d71c2e30-4f82-40d8-810a-d80c77d1752a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:27.385481 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d71c2e30-4f82-40d8-810a-d80c77d1752a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Going to try to stop instance {{(pid=124604) force_stop /opt/stack/nova/nova/compute/api.py:2707}} May 28 03:39:27.561180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d71c2e30-4f82-40d8-810a-d80c77d1752a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "POST /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/action" status: 202 len: 0 microversion: 2.1 time: 0.248625 May 28 03:39:27.561810 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 250/487] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:39:27 2022] POST /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/action => generated 0 bytes in 250 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:39:27.577044 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6da50820-b36e-430c-98f3-04c62afe23ff tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:27.577918 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6da50820-b36e-430c-98f3-04c62afe23ff tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:27.584238 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6da50820-b36e-430c-98f3-04c62afe23ff tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:27.584686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6da50820-b36e-430c-98f3-04c62afe23ff tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:27.585491 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6da50820-b36e-430c-98f3-04c62afe23ff tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:27.585938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6da50820-b36e-430c-98f3-04c62afe23ff tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:27.716363 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6da50820-b36e-430c-98f3-04c62afe23ff tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:27.723878 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6da50820-b36e-430c-98f3-04c62afe23ff tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.151571 May 28 03:39:27.724442 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 238/488] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:27 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:28.744093 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-62074c76-4ca5-4d23-ab80-db242ae8583b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:28.744519 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-62074c76-4ca5-4d23-ab80-db242ae8583b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:28.750810 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-62074c76-4ca5-4d23-ab80-db242ae8583b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:28.751299 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-62074c76-4ca5-4d23-ab80-db242ae8583b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:28.752177 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-62074c76-4ca5-4d23-ab80-db242ae8583b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:28.752614 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-62074c76-4ca5-4d23-ab80-db242ae8583b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:28.884487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-62074c76-4ca5-4d23-ab80-db242ae8583b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:28.892231 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-62074c76-4ca5-4d23-ab80-db242ae8583b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.154609 May 28 03:39:28.892802 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 251/489] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:28 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:29.915913 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-977741bb-754c-4e46-89be-efc7dc4d0538 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:29.916611 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-977741bb-754c-4e46-89be-efc7dc4d0538 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:29.923212 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-977741bb-754c-4e46-89be-efc7dc4d0538 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:29.923656 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-977741bb-754c-4e46-89be-efc7dc4d0538 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:29.924447 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-977741bb-754c-4e46-89be-efc7dc4d0538 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:29.924871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-977741bb-754c-4e46-89be-efc7dc4d0538 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:30.070572 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-977741bb-754c-4e46-89be-efc7dc4d0538 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:30.077354 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-977741bb-754c-4e46-89be-efc7dc4d0538 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.171419 May 28 03:39:30.078026 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 239/490] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:29 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:30.382653 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dd4ff9c7-d091-4216-98ce-063ca055ea86 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:30.383722 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dd4ff9c7-d091-4216-98ce-063ca055ea86 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:30.390842 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd4ff9c7-d091-4216-98ce-063ca055ea86 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:30.391293 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd4ff9c7-d091-4216-98ce-063ca055ea86 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:30.392052 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd4ff9c7-d091-4216-98ce-063ca055ea86 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:30.392483 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd4ff9c7-d091-4216-98ce-063ca055ea86 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:30.416639 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-dd4ff9c7-d091-4216-98ce-063ca055ea86 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8/ips" status: 200 len: 149 microversion: 2.1 time: 0.038321 May 28 03:39:30.418498 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 252/491] 10.222.0.41 () {58 vars in 1247 bytes} [Sat May 28 03:39:30 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8/ips => generated 149 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:39:30.428313 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5a4da5ec-1fd3-49f0-9caa-af2c244ff601 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cdb718c8-6afe-412f-b14b-ddce4cf83a7a"}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:30.429459 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5a4da5ec-1fd3-49f0-9caa-af2c244ff601 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:30.436120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5a4da5ec-1fd3-49f0-9caa-af2c244ff601 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:30.436658 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5a4da5ec-1fd3-49f0-9caa-af2c244ff601 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:30.437346 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5a4da5ec-1fd3-49f0-9caa-af2c244ff601 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:30.437780 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5a4da5ec-1fd3-49f0-9caa-af2c244ff601 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:30.524566 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-5a4da5ec-1fd3-49f0-9caa-af2c244ff601 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Invalid volume: volume cdb718c8-6afe-412f-b14b-ddce4cf83a7a is already attached to instances: 161bde4a-7b5a-4c34-9cad-add4855d9d1f May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5a4da5ec-1fd3-49f0-9caa-af2c244ff601 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: POST /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8/os-volume_attachments HTTP/1.1 May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 74 May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:39:30.526534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {"volumeAttachment": {"volumeId": "cdb718c8-6afe-412f-b14b-ddce4cf83a7a"}} May 28 03:39:30.527966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Invalid volume: volume cdb718c8-6afe-412f-b14b-ddce4cf83a7a is already attached to instances: 161bde4a-7b5a-4c34-9cad-add4855d9d1f May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'server_id': '8714c46b-f821-4c0a-b55d-1083da5eb5b8', 'body': {'volumeAttachment': {'volumeId': 'cdb718c8-6afe-412f-b14b-ddce4cf83a7a'}}} May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume cdb718c8-6afe-412f-b14b-ddce4cf83a7a is already attached to instances: 161bde4a-7b5a-4c34-9cad-add4855d9d1f May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:30.528706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume cdb718c8-6afe-412f-b14b-ddce4cf83a7a is already attached to instances: 161bde4a-7b5a-4c34-9cad-add4855d9d1f May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:30.529865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:39:30.531066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:39:30.531066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:39:30.531066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:39:30.531066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume cdb718c8-6afe-412f-b14b-ddce4cf83a7a is already attached to instances: 161bde4a-7b5a-4c34-9cad-add4855d9d1f May 28 03:39:30.531066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:30.531066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5a4da5ec-1fd3-49f0-9caa-af2c244ff601 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 400 to user: Invalid volume: volume cdb718c8-6afe-412f-b14b-ddce4cf83a7a is already attached to instances: 161bde4a-7b5a-4c34-9cad-add4855d9d1f {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:39:30.531066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5a4da5ec-1fd3-49f0-9caa-af2c244ff601 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.103758 May 28 03:39:30.531066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 240/492] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:39:30 2022] POST /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8/os-volume_attachments => generated 174 bytes in 105 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 28 03:39:30.537097 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3cec196a-6cba-4a8d-bf55-6bc529beb745 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:30.543489 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3cec196a-6cba-4a8d-bf55-6bc529beb745 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:30.544571 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3cec196a-6cba-4a8d-bf55-6bc529beb745 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:30.545036 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3cec196a-6cba-4a8d-bf55-6bc529beb745 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:30.545968 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3cec196a-6cba-4a8d-bf55-6bc529beb745 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:30.546403 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3cec196a-6cba-4a8d-bf55-6bc529beb745 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:30.571510 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3cec196a-6cba-4a8d-bf55-6bc529beb745 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Going to try to terminate instance {{(pid=124604) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:39:30.839180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3cec196a-6cba-4a8d-bf55-6bc529beb745 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 204 len: 0 microversion: 2.1 time: 0.305669 May 28 03:39:30.839841 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 253/493] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:39:30 2022] DELETE /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:39:30.848591 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a3b3d706-5cdd-4ddb-b247-a1fce458827b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:30.849226 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a3b3d706-5cdd-4ddb-b247-a1fce458827b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:30.859739 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3b3d706-5cdd-4ddb-b247-a1fce458827b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:30.860184 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3b3d706-5cdd-4ddb-b247-a1fce458827b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:30.861079 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3b3d706-5cdd-4ddb-b247-a1fce458827b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:30.866975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3b3d706-5cdd-4ddb-b247-a1fce458827b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:30.992370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a3b3d706-5cdd-4ddb-b247-a1fce458827b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:31.004769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a3b3d706-5cdd-4ddb-b247-a1fce458827b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1680 microversion: 2.1 time: 0.160314 May 28 03:39:31.013199 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 241/494] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:30 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1680 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:31.095022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9b4ce8bf-8485-46cf-b637-7fdbc23afa97 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:31.095725 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9b4ce8bf-8485-46cf-b637-7fdbc23afa97 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:31.101619 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b4ce8bf-8485-46cf-b637-7fdbc23afa97 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:31.102636 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b4ce8bf-8485-46cf-b637-7fdbc23afa97 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:31.103403 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b4ce8bf-8485-46cf-b637-7fdbc23afa97 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:31.103842 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9b4ce8bf-8485-46cf-b637-7fdbc23afa97 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:31.235620 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9b4ce8bf-8485-46cf-b637-7fdbc23afa97 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:31.243348 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9b4ce8bf-8485-46cf-b637-7fdbc23afa97 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.152655 May 28 03:39:31.243921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 254/495] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:31 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:32.024996 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c07e7e80-6d45-4686-ad75-bc5378e7d9e8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:32.025632 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c07e7e80-6d45-4686-ad75-bc5378e7d9e8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:32.032051 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c07e7e80-6d45-4686-ad75-bc5378e7d9e8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:32.032487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c07e7e80-6d45-4686-ad75-bc5378e7d9e8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:32.033277 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c07e7e80-6d45-4686-ad75-bc5378e7d9e8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:32.033706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c07e7e80-6d45-4686-ad75-bc5378e7d9e8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:32.167888 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c07e7e80-6d45-4686-ad75-bc5378e7d9e8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:32.175369 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c07e7e80-6d45-4686-ad75-bc5378e7d9e8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1680 microversion: 2.1 time: 0.154216 May 28 03:39:32.175924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 242/496] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:32 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:32.260387 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-702df575-46b9-4c5f-92d9-5b82abe2287e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:32.261087 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-702df575-46b9-4c5f-92d9-5b82abe2287e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:32.268001 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-702df575-46b9-4c5f-92d9-5b82abe2287e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:32.268524 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-702df575-46b9-4c5f-92d9-5b82abe2287e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:32.269413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-702df575-46b9-4c5f-92d9-5b82abe2287e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:32.269932 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-702df575-46b9-4c5f-92d9-5b82abe2287e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:32.407555 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-702df575-46b9-4c5f-92d9-5b82abe2287e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:32.415316 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-702df575-46b9-4c5f-92d9-5b82abe2287e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.159259 May 28 03:39:32.415958 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 255/497] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:32 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:33.193455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-70b14384-d357-4c8b-95e6-491df3ed7e14 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:33.193998 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-70b14384-d357-4c8b-95e6-491df3ed7e14 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:33.200969 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-70b14384-d357-4c8b-95e6-491df3ed7e14 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:33.201444 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-70b14384-d357-4c8b-95e6-491df3ed7e14 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:33.202201 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-70b14384-d357-4c8b-95e6-491df3ed7e14 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:33.202632 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-70b14384-d357-4c8b-95e6-491df3ed7e14 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:33.349058 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-70b14384-d357-4c8b-95e6-491df3ed7e14 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:33.357545 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-70b14384-d357-4c8b-95e6-491df3ed7e14 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1680 microversion: 2.1 time: 0.168103 May 28 03:39:33.358245 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 243/498] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:33 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1680 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:33.431803 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-549b2ca3-77c3-4cf6-8ba1-1e0b9982537f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:33.432420 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-549b2ca3-77c3-4cf6-8ba1-1e0b9982537f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:33.439473 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-549b2ca3-77c3-4cf6-8ba1-1e0b9982537f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:33.450935 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-549b2ca3-77c3-4cf6-8ba1-1e0b9982537f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:33.451645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-549b2ca3-77c3-4cf6-8ba1-1e0b9982537f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:33.452095 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-549b2ca3-77c3-4cf6-8ba1-1e0b9982537f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:33.595683 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-549b2ca3-77c3-4cf6-8ba1-1e0b9982537f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:33.603298 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-549b2ca3-77c3-4cf6-8ba1-1e0b9982537f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.175446 May 28 03:39:33.603853 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 256/499] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:33 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:34.305799 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-3d72c68c-8dde-41db-94f4-c6878dd2f308 req-8250b9bd-ff30-4d12-836f-9ef44f27ed1f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8714c46b-f821-4c0a-b55d-1083da5eb5b8", "name": "network-vif-deleted", "tag": "bce2d221-cb90-4b9d-b2ca-1b1020b1c795"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:34.314824 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-3d72c68c-8dde-41db-94f4-c6878dd2f308 req-8250b9bd-ff30-4d12-836f-9ef44f27ed1f service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:34.315511 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-3d72c68c-8dde-41db-94f4-c6878dd2f308 req-8250b9bd-ff30-4d12-836f-9ef44f27ed1f service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:34.333774 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-3d72c68c-8dde-41db-94f4-c6878dd2f308 req-8250b9bd-ff30-4d12-836f-9ef44f27ed1f service nova] Creating event network-vif-deleted:bce2d221-cb90-4b9d-b2ca-1b1020b1c795 for instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 on hv-cinder-84336 May 28 03:39:34.339281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-3d72c68c-8dde-41db-94f4-c6878dd2f308 req-8250b9bd-ff30-4d12-836f-9ef44f27ed1f service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037149 May 28 03:39:34.340238 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 244/500] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:39:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:39:34.375200 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-50fdbaea-76a7-4d1e-b0b6-6dc6a66bea6d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:34.376075 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-50fdbaea-76a7-4d1e-b0b6-6dc6a66bea6d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:34.382409 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-50fdbaea-76a7-4d1e-b0b6-6dc6a66bea6d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:34.382799 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-50fdbaea-76a7-4d1e-b0b6-6dc6a66bea6d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:34.384438 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-50fdbaea-76a7-4d1e-b0b6-6dc6a66bea6d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:34.384691 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-50fdbaea-76a7-4d1e-b0b6-6dc6a66bea6d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:34.465081 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-50fdbaea-76a7-4d1e-b0b6-6dc6a66bea6d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:34.472908 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-50fdbaea-76a7-4d1e-b0b6-6dc6a66bea6d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 200 len: 1612 microversion: 2.1 time: 0.101804 May 28 03:39:34.476893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 257/501] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:34 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 1612 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:34.621000 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5872af5d-cbb2-4225-a1f7-d0486253d997 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:34.621866 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5872af5d-cbb2-4225-a1f7-d0486253d997 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:34.631650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5872af5d-cbb2-4225-a1f7-d0486253d997 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:34.632594 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5872af5d-cbb2-4225-a1f7-d0486253d997 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:34.634205 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5872af5d-cbb2-4225-a1f7-d0486253d997 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:34.634866 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5872af5d-cbb2-4225-a1f7-d0486253d997 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:34.816227 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5872af5d-cbb2-4225-a1f7-d0486253d997 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:34.824479 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5872af5d-cbb2-4225-a1f7-d0486253d997 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.208106 May 28 03:39:34.825396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 245/502] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:34 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:35.490611 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9a66e86e-02d9-47f0-b2c9-42ad0b32460c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:35.491395 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9a66e86e-02d9-47f0-b2c9-42ad0b32460c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:35.498146 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9a66e86e-02d9-47f0-b2c9-42ad0b32460c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:35.498619 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9a66e86e-02d9-47f0-b2c9-42ad0b32460c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:35.499532 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9a66e86e-02d9-47f0-b2c9-42ad0b32460c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:35.499970 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9a66e86e-02d9-47f0-b2c9-42ad0b32460c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:35.513840 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-9a66e86e-02d9-47f0-b2c9-42ad0b32460c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9a66e86e-02d9-47f0-b2c9-42ad0b32460c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 HTTP/1.1 May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:39:35.515302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:39:35.516825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': '8714c46b-f821-4c0a-b55d-1083da5eb5b8'} May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:39:35.517796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:39:35.519337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:39:35.521374 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9a66e86e-02d9-47f0-b2c9-42ad0b32460c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:39:35.521374 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9a66e86e-02d9-47f0-b2c9-42ad0b32460c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 404 len: 111 microversion: 2.1 time: 0.030380 May 28 03:39:35.521374 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 258/503] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:35 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:39:35.526154 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-10c7458a-0284-4116-adbe-04d3992fe80a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '.version_select at 0x7feb1fd10040>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:35.594048 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-10c7458a-0284-4116-adbe-04d3992fe80a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1533452588" status: 202 len: 0 microversion: 2.1 time: 0.072157 May 28 03:39:35.594671 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 246/504] 10.222.0.41 () {58 vars in 1220 bytes} [Sat May 28 03:39:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1533452588 => generated 0 bytes in 73 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:39:35.844078 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-60ce0714-2628-4738-812b-caffdf26ae69 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:35.844835 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-60ce0714-2628-4738-812b-caffdf26ae69 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:35.850435 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-02ce1593-0d7d-4988-8d2d-d360353d23e2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:35.850763 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-02ce1593-0d7d-4988-8d2d-d360353d23e2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:35.866607 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-60ce0714-2628-4738-812b-caffdf26ae69 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:35.867607 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-60ce0714-2628-4738-812b-caffdf26ae69 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:35.869489 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-60ce0714-2628-4738-812b-caffdf26ae69 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:35.870366 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-60ce0714-2628-4738-812b-caffdf26ae69 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:35.871931 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02ce1593-0d7d-4988-8d2d-d360353d23e2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:35.873432 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02ce1593-0d7d-4988-8d2d-d360353d23e2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:35.883963 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02ce1593-0d7d-4988-8d2d-d360353d23e2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:35.884414 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02ce1593-0d7d-4988-8d2d-d360353d23e2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:36.032868 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-60ce0714-2628-4738-812b-caffdf26ae69 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lazy-loading 'services' on Instance uuid e654573a-25d3-4e44-b150-e0fdd2ef00df {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:39:36.074595 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-02ce1593-0d7d-4988-8d2d-d360353d23e2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:36.082881 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-02ce1593-0d7d-4988-8d2d-d360353d23e2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.245123 May 28 03:39:36.092940 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 259/505] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:35 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:36.256484 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-60ce0714-2628-4738-812b-caffdf26ae69 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "DELETE /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df/os-volume_attachments/89be344a-1f24-4f89-b3fb-aa83f3d2f392" status: 202 len: 0 microversion: 2.1 time: 0.417596 May 28 03:39:36.257485 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 247/506] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:39:35 2022] DELETE /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df/os-volume_attachments/89be344a-1f24-4f89-b3fb-aa83f3d2f392 => generated 0 bytes in 420 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:39:36.832527 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4c0d5b87-c127-4c18-a402-d3bd5e308fe2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:36.833041 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4c0d5b87-c127-4c18-a402-d3bd5e308fe2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:36.839394 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c0d5b87-c127-4c18-a402-d3bd5e308fe2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:36.839838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c0d5b87-c127-4c18-a402-d3bd5e308fe2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:36.840594 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c0d5b87-c127-4c18-a402-d3bd5e308fe2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:36.841023 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c0d5b87-c127-4c18-a402-d3bd5e308fe2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:36.969287 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-4c0d5b87-c127-4c18-a402-d3bd5e308fe2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lazy-loading 'services' on Instance uuid 161bde4a-7b5a-4c34-9cad-add4855d9d1f {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:39:37.100959 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ca904e84-f1d6-4255-bbb3-e9edc2151eee tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:37.101743 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ca904e84-f1d6-4255-bbb3-e9edc2151eee tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:37.108927 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ca904e84-f1d6-4255-bbb3-e9edc2151eee tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:37.109424 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ca904e84-f1d6-4255-bbb3-e9edc2151eee tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:37.110198 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ca904e84-f1d6-4255-bbb3-e9edc2151eee tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:37.110667 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ca904e84-f1d6-4255-bbb3-e9edc2151eee tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:37.227207 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4c0d5b87-c127-4c18-a402-d3bd5e308fe2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments/cdb718c8-6afe-412f-b14b-ddce4cf83a7a" status: 202 len: 0 microversion: 2.1 time: 0.397632 May 28 03:39:37.228002 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 260/507] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:39:36 2022] DELETE /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments/cdb718c8-6afe-412f-b14b-ddce4cf83a7a => generated 0 bytes in 400 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:39:37.236933 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1f5956a2-92dc-4511-be9a-aa8dcc9607d8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:37.238281 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1f5956a2-92dc-4511-be9a-aa8dcc9607d8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:37.249291 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f5956a2-92dc-4511-be9a-aa8dcc9607d8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:37.249996 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f5956a2-92dc-4511-be9a-aa8dcc9607d8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:37.250777 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f5956a2-92dc-4511-be9a-aa8dcc9607d8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:37.251206 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f5956a2-92dc-4511-be9a-aa8dcc9607d8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:37.259412 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ca904e84-f1d6-4255-bbb3-e9edc2151eee tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:37.269053 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ca904e84-f1d6-4255-bbb3-e9edc2151eee tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.172662 May 28 03:39:37.272564 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 248/508] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:37 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:39:37.281673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1f5956a2-92dc-4511-be9a-aa8dcc9607d8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049146 May 28 03:39:37.282221 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 261/509] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:39:37 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:39:37.404955 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-06272f6e-75b1-451c-8eff-349d0a57e554 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:37.405503 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-06272f6e-75b1-451c-8eff-349d0a57e554 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:37.411793 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06272f6e-75b1-451c-8eff-349d0a57e554 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:37.412237 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06272f6e-75b1-451c-8eff-349d0a57e554 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:37.413024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06272f6e-75b1-451c-8eff-349d0a57e554 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:37.413844 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-06272f6e-75b1-451c-8eff-349d0a57e554 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:37.445010 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-06272f6e-75b1-451c-8eff-349d0a57e554 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:39:37.741636 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-06272f6e-75b1-451c-8eff-349d0a57e554 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "DELETE /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 204 len: 0 microversion: 2.1 time: 0.340338 May 28 03:39:37.742111 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 249/510] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:39:37 2022] DELETE /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 0 bytes in 342 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:39:37.750824 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f47fb5b4-dd68-47ec-b98b-c7e7fa8e54e2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:37.751603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f47fb5b4-dd68-47ec-b98b-c7e7fa8e54e2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:37.759447 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f47fb5b4-dd68-47ec-b98b-c7e7fa8e54e2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:37.759978 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f47fb5b4-dd68-47ec-b98b-c7e7fa8e54e2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:37.760749 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f47fb5b4-dd68-47ec-b98b-c7e7fa8e54e2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:37.761259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f47fb5b4-dd68-47ec-b98b-c7e7fa8e54e2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:37.894305 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f47fb5b4-dd68-47ec-b98b-c7e7fa8e54e2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:37.902569 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f47fb5b4-dd68-47ec-b98b-c7e7fa8e54e2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1514 microversion: 2.1 time: 0.155695 May 28 03:39:37.903334 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 262/511] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:37 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1514 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:38.289013 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a8cd559b-0cb5-4e40-9533-18f9d033e2ec tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:38.290244 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a8cd559b-0cb5-4e40-9533-18f9d033e2ec tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:38.293392 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-68167a43-b779-4960-9edc-463a4000f1ec tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:38.294443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-68167a43-b779-4960-9edc-463a4000f1ec tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:38.296470 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a8cd559b-0cb5-4e40-9533-18f9d033e2ec tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:38.296922 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a8cd559b-0cb5-4e40-9533-18f9d033e2ec tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:38.297716 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a8cd559b-0cb5-4e40-9533-18f9d033e2ec tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:38.298154 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a8cd559b-0cb5-4e40-9533-18f9d033e2ec tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:38.300300 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68167a43-b779-4960-9edc-463a4000f1ec tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:38.300741 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68167a43-b779-4960-9edc-463a4000f1ec tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:38.301519 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68167a43-b779-4960-9edc-463a4000f1ec tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:38.301948 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68167a43-b779-4960-9edc-463a4000f1ec tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:38.334462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-68167a43-b779-4960-9edc-463a4000f1ec tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045350 May 28 03:39:38.335027 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 263/512] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:39:38 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:39:38.433022 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a8cd559b-0cb5-4e40-9533-18f9d033e2ec tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:38.440384 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a8cd559b-0cb5-4e40-9533-18f9d033e2ec tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.155572 May 28 03:39:38.441125 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 250/513] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:38 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:38.919811 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a1016348-05b5-49f3-ac74-135350dac548 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:38.920414 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a1016348-05b5-49f3-ac74-135350dac548 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:38.926842 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a1016348-05b5-49f3-ac74-135350dac548 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:38.927317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a1016348-05b5-49f3-ac74-135350dac548 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:38.928070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a1016348-05b5-49f3-ac74-135350dac548 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:38.928504 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a1016348-05b5-49f3-ac74-135350dac548 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:38.932628 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:39.061450 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a1016348-05b5-49f3-ac74-135350dac548 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:39.068819 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a1016348-05b5-49f3-ac74-135350dac548 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1514 microversion: 2.1 time: 0.152871 May 28 03:39:39.069424 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 264/514] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:38 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1514 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:39.347858 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7d7880f1-10f6-40c2-9991-8387dbe825b7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:39.349074 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7d7880f1-10f6-40c2-9991-8387dbe825b7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:39.356656 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d7880f1-10f6-40c2-9991-8387dbe825b7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:39.357131 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d7880f1-10f6-40c2-9991-8387dbe825b7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:39.358136 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d7880f1-10f6-40c2-9991-8387dbe825b7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:39.358565 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d7880f1-10f6-40c2-9991-8387dbe825b7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:39.388663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7d7880f1-10f6-40c2-9991-8387dbe825b7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045841 May 28 03:39:39.389244 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 251/515] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:39:39 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:39:39.457332 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-31ecb966-816b-431c-a433-cee4a0407880 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:39.458080 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-31ecb966-816b-431c-a433-cee4a0407880 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:39.464135 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-31ecb966-816b-431c-a433-cee4a0407880 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:39.464589 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-31ecb966-816b-431c-a433-cee4a0407880 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:39.465377 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-31ecb966-816b-431c-a433-cee4a0407880 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:39.465817 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-31ecb966-816b-431c-a433-cee4a0407880 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:39.602450 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-31ecb966-816b-431c-a433-cee4a0407880 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:39.610129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-31ecb966-816b-431c-a433-cee4a0407880 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.156690 May 28 03:39:39.610716 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 265/516] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:39 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:40.087811 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9f39c583-ec60-49f8-84da-46f60d8c07f0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:40.088448 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9f39c583-ec60-49f8-84da-46f60d8c07f0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:40.094905 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9f39c583-ec60-49f8-84da-46f60d8c07f0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:40.095418 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9f39c583-ec60-49f8-84da-46f60d8c07f0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:40.096422 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9f39c583-ec60-49f8-84da-46f60d8c07f0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:40.096864 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9f39c583-ec60-49f8-84da-46f60d8c07f0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:40.101366 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:40.303149 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9f39c583-ec60-49f8-84da-46f60d8c07f0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:40.310936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9f39c583-ec60-49f8-84da-46f60d8c07f0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1514 microversion: 2.1 time: 0.227700 May 28 03:39:40.311502 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 252/517] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:40 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1514 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:40.400924 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9952ffc8-27b9-4883-8005-7b65d84ca2d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:40.401939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9952ffc8-27b9-4883-8005-7b65d84ca2d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:40.408650 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9952ffc8-27b9-4883-8005-7b65d84ca2d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:40.409096 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9952ffc8-27b9-4883-8005-7b65d84ca2d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:40.409908 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9952ffc8-27b9-4883-8005-7b65d84ca2d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:40.410343 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9952ffc8-27b9-4883-8005-7b65d84ca2d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:40.438319 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9952ffc8-27b9-4883-8005-7b65d84ca2d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041790 May 28 03:39:40.439009 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 266/518] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:39:40 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 28 03:39:40.507100 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4bbe3793-25a9-490e-89cb-b7a1bb56868c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:40.507599 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4bbe3793-25a9-490e-89cb-b7a1bb56868c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:40.514607 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4bbe3793-25a9-490e-89cb-b7a1bb56868c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:40.515050 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4bbe3793-25a9-490e-89cb-b7a1bb56868c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:40.515809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4bbe3793-25a9-490e-89cb-b7a1bb56868c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:40.516247 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4bbe3793-25a9-490e-89cb-b7a1bb56868c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:40.547390 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4bbe3793-25a9-490e-89cb-b7a1bb56868c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:39:40.610928 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-c68425d9-ecd0-4ed6-8c06-5cf3e4481ee8 req-f770cb48-a822-4974-91f6-4a3ce993ed84 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e654573a-25d3-4e44-b150-e0fdd2ef00df", "name": "network-vif-deleted", "tag": "86370b2e-5f98-49df-b01e-c1a775a9facd"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:40.620702 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-c68425d9-ecd0-4ed6-8c06-5cf3e4481ee8 req-f770cb48-a822-4974-91f6-4a3ce993ed84 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:40.621156 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-c68425d9-ecd0-4ed6-8c06-5cf3e4481ee8 req-f770cb48-a822-4974-91f6-4a3ce993ed84 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:40.632513 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-c68425d9-ecd0-4ed6-8c06-5cf3e4481ee8 req-f770cb48-a822-4974-91f6-4a3ce993ed84 service nova] Creating event network-vif-deleted:86370b2e-5f98-49df-b01e-c1a775a9facd for instance e654573a-25d3-4e44-b150-e0fdd2ef00df on hv-cinder-84336 May 28 03:39:40.637942 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-c68425d9-ecd0-4ed6-8c06-5cf3e4481ee8 req-f770cb48-a822-4974-91f6-4a3ce993ed84 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030626 May 28 03:39:40.638600 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 267/519] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:39:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:39:40.645257 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bd803878-2f50-4ad6-8477-fdef3d2c6f92 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:40.655866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bd803878-2f50-4ad6-8477-fdef3d2c6f92 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:40.655866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd803878-2f50-4ad6-8477-fdef3d2c6f92 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:40.655866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd803878-2f50-4ad6-8477-fdef3d2c6f92 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:40.655866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd803878-2f50-4ad6-8477-fdef3d2c6f92 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:40.656625 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd803878-2f50-4ad6-8477-fdef3d2c6f92 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:40.806078 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bd803878-2f50-4ad6-8477-fdef3d2c6f92 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:40.814033 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bd803878-2f50-4ad6-8477-fdef3d2c6f92 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.173161 May 28 03:39:40.814611 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 268/520] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:40 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:40.820332 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4bbe3793-25a9-490e-89cb-b7a1bb56868c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 204 len: 0 microversion: 2.1 time: 0.316890 May 28 03:39:40.821774 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 253/521] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:39:40 2022] DELETE /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 28 03:39:40.834256 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-cf033d4e-f6cc-413e-812d-01cfa818be6b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:40.834872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-cf033d4e-f6cc-413e-812d-01cfa818be6b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:40.840911 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cf033d4e-f6cc-413e-812d-01cfa818be6b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:40.841392 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cf033d4e-f6cc-413e-812d-01cfa818be6b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:40.842163 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cf033d4e-f6cc-413e-812d-01cfa818be6b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:40.842599 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cf033d4e-f6cc-413e-812d-01cfa818be6b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:40.975415 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-cf033d4e-f6cc-413e-812d-01cfa818be6b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:40.983329 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-cf033d4e-f6cc-413e-812d-01cfa818be6b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1679 microversion: 2.1 time: 0.152970 May 28 03:39:40.983890 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 269/522] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:40 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1679 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:41.329362 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0683d9a8-442a-496f-b9e7-f2e1f248ac82 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:41.330075 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0683d9a8-442a-496f-b9e7-f2e1f248ac82 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:41.336594 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0683d9a8-442a-496f-b9e7-f2e1f248ac82 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:41.337034 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0683d9a8-442a-496f-b9e7-f2e1f248ac82 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:41.337817 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0683d9a8-442a-496f-b9e7-f2e1f248ac82 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:41.338243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0683d9a8-442a-496f-b9e7-f2e1f248ac82 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:41.369717 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0683d9a8-442a-496f-b9e7-f2e1f248ac82 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:41.370155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0683d9a8-442a-496f-b9e7-f2e1f248ac82 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:41.370609 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-0683d9a8-442a-496f-b9e7-f2e1f248ac82 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lazy-loading 'fault' on Instance uuid e654573a-25d3-4e44-b150-e0fdd2ef00df {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:39:41.411777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0683d9a8-442a-496f-b9e7-f2e1f248ac82 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:41.419773 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0683d9a8-442a-496f-b9e7-f2e1f248ac82 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1309 microversion: 2.1 time: 0.094707 May 28 03:39:41.420364 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 254/523] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:41 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1309 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:41.831584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-801006d5-efe9-421c-8765-4de78e9fb6d7 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:41.832193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-801006d5-efe9-421c-8765-4de78e9fb6d7 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:41.839460 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-801006d5-efe9-421c-8765-4de78e9fb6d7 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:41.839888 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-801006d5-efe9-421c-8765-4de78e9fb6d7 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:41.840827 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-801006d5-efe9-421c-8765-4de78e9fb6d7 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:41.841152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-801006d5-efe9-421c-8765-4de78e9fb6d7 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:41.993592 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-801006d5-efe9-421c-8765-4de78e9fb6d7 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:42.001280 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-801006d5-efe9-421c-8765-4de78e9fb6d7 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.173416 May 28 03:39:42.001665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1ffa10bd-f106-4568-9245-f82aeeeaaeb0 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:42.002103 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 270/524] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:41 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:42.002361 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1ffa10bd-f106-4568-9245-f82aeeeaaeb0 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:42.009700 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ffa10bd-f106-4568-9245-f82aeeeaaeb0 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:42.010137 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ffa10bd-f106-4568-9245-f82aeeeaaeb0 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:42.010894 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ffa10bd-f106-4568-9245-f82aeeeaaeb0 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:42.011322 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ffa10bd-f106-4568-9245-f82aeeeaaeb0 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:42.148805 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1ffa10bd-f106-4568-9245-f82aeeeaaeb0 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:42.156292 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1ffa10bd-f106-4568-9245-f82aeeeaaeb0 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1679 microversion: 2.1 time: 0.158443 May 28 03:39:42.156848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 255/525] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:41 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1679 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:39:42.437501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bd075e69-ba93-4886-b78f-e079be1394ac tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:42.438028 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bd075e69-ba93-4886-b78f-e079be1394ac tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:42.444160 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd075e69-ba93-4886-b78f-e079be1394ac tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:42.444614 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd075e69-ba93-4886-b78f-e079be1394ac tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:42.445465 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd075e69-ba93-4886-b78f-e079be1394ac tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:42.445900 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd075e69-ba93-4886-b78f-e079be1394ac tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:42.476181 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd075e69-ba93-4886-b78f-e079be1394ac tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:42.477186 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bd075e69-ba93-4886-b78f-e079be1394ac tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:42.477613 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-bd075e69-ba93-4886-b78f-e079be1394ac tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lazy-loading 'fault' on Instance uuid e654573a-25d3-4e44-b150-e0fdd2ef00df {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:39:42.522717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bd075e69-ba93-4886-b78f-e079be1394ac tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:42.530188 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bd075e69-ba93-4886-b78f-e079be1394ac tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 200 len: 1309 microversion: 2.1 time: 0.096733 May 28 03:39:42.530745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 271/526] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:42 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 1309 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:43.019812 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-87bc310a-9aca-4718-9548-13a651504807 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:43.020514 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-87bc310a-9aca-4718-9548-13a651504807 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:43.028945 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87bc310a-9aca-4718-9548-13a651504807 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:43.038206 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87bc310a-9aca-4718-9548-13a651504807 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:43.038206 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87bc310a-9aca-4718-9548-13a651504807 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:43.038206 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87bc310a-9aca-4718-9548-13a651504807 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:43.173965 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b1fa79d5-8981-4e29-9e79-43c44e92258f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:43.174620 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b1fa79d5-8981-4e29-9e79-43c44e92258f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:43.183743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b1fa79d5-8981-4e29-9e79-43c44e92258f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:43.184190 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b1fa79d5-8981-4e29-9e79-43c44e92258f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:43.184997 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b1fa79d5-8981-4e29-9e79-43c44e92258f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:43.185463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b1fa79d5-8981-4e29-9e79-43c44e92258f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:43.270198 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-87bc310a-9aca-4718-9548-13a651504807 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:43.278627 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-87bc310a-9aca-4718-9548-13a651504807 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1677 microversion: 2.1 time: 0.262928 May 28 03:39:43.288397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 256/527] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:43 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1677 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:43.327813 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b1fa79d5-8981-4e29-9e79-43c44e92258f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:43.335603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b1fa79d5-8981-4e29-9e79-43c44e92258f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1679 microversion: 2.1 time: 0.165841 May 28 03:39:43.336181 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 272/528] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:43 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1679 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:43.547504 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1988f91b-dafd-436c-a501-9a4e82a08c87 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:43.548019 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1988f91b-dafd-436c-a501-9a4e82a08c87 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: e654573a-25d3-4e44-b150-e0fdd2ef00df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:43.554220 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1988f91b-dafd-436c-a501-9a4e82a08c87 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:43.554671 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1988f91b-dafd-436c-a501-9a4e82a08c87 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:43.556038 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1988f91b-dafd-436c-a501-9a4e82a08c87 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:43.556480 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1988f91b-dafd-436c-a501-9a4e82a08c87 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:43.572253 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-1988f91b-dafd-436c-a501-9a4e82a08c87 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] HTTP exception thrown: Instance e654573a-25d3-4e44-b150-e0fdd2ef00df could not be found. May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1988f91b-dafd-436c-a501-9a4e82a08c87 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Request method failure captured: May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df HTTP/1.1 May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZknNw6OgVOskXFoQ_7IdZXggvUl-unozmFQ1gLcu8_rV9Si5p65dR_LKNqmNofwjPsnv7CWUdOlv9gXVmJrrTITvBGr_I96CH6HA8h_mLgkldw7kHlMlioYEk5Bg8nj2-roWp6ZC6_tVCMbLwFEWkmTbMWtfbMtd9J510X3jUpxWmw May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: e302a2cd63ba46d7a6cef952ddb89242 May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-2104603578 May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:39:43.574521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/e302a2cd63ba46d7a6cef952ddb89242"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/e302a2cd63ba46d7a6cef952ddb89242"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/e302a2cd63ba46d7a6cef952ddb89242"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_e302a2cd63ba46d7a6cef952ddb89242"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-VolumesSnapshotTestJSON-2104603578 May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: e302a2cd63ba46d7a6cef952ddb89242 May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-2104603578 May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-VolumesSnapshotTestJSON-2104603578-project May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: af5fa544bc6440c9830d25e4ad4d5987 May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-VolumesSnapshotTestJSON-2104603578-project May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance e654573a-25d3-4e44-b150-e0fdd2ef00df could not be found. May 28 03:39:43.576138 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': 'e654573a-25d3-4e44-b150-e0fdd2ef00df'} May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e654573a-25d3-4e44-b150-e0fdd2ef00df could not be found. May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:39:43.576930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e654573a-25d3-4e44-b150-e0fdd2ef00df could not be found. May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e654573a-25d3-4e44-b150-e0fdd2ef00df could not be found. May 28 03:39:43.578225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:43.579275 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1988f91b-dafd-436c-a501-9a4e82a08c87 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Returning 404 to user: Instance e654573a-25d3-4e44-b150-e0fdd2ef00df could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:39:43.579275 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1988f91b-dafd-436c-a501-9a4e82a08c87 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df" status: 404 len: 111 microversion: 2.1 time: 0.032549 May 28 03:39:43.579275 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 257/529] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:43 2022] GET /compute/v2.1/servers/e654573a-25d3-4e44-b150-e0fdd2ef00df => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:39:43.816250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-8df137a9-9721-4909-a7d3-9c644b5bd427 req-bbb60b15-87f4-4890-87b6-abb4de6cc4a1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "161bde4a-7b5a-4c34-9cad-add4855d9d1f", "name": "network-vif-deleted", "tag": "a44b7ed2-6364-44d9-9fbc-d45c00c0413e"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:43.824428 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-8df137a9-9721-4909-a7d3-9c644b5bd427 req-bbb60b15-87f4-4890-87b6-abb4de6cc4a1 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:43.824859 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-8df137a9-9721-4909-a7d3-9c644b5bd427 req-bbb60b15-87f4-4890-87b6-abb4de6cc4a1 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:43.836916 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-8df137a9-9721-4909-a7d3-9c644b5bd427 req-bbb60b15-87f4-4890-87b6-abb4de6cc4a1 service nova] Creating event network-vif-deleted:a44b7ed2-6364-44d9-9fbc-d45c00c0413e for instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f on hv-cinder-84336 May 28 03:39:43.842769 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-8df137a9-9721-4909-a7d3-9c644b5bd427 req-bbb60b15-87f4-4890-87b6-abb4de6cc4a1 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030065 May 28 03:39:43.843601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 273/530] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:39:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:39:44.295762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3db2a778-0d7d-4dac-8bd3-4e19b1b08587 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:44.296379 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3db2a778-0d7d-4dac-8bd3-4e19b1b08587 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:44.302749 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3db2a778-0d7d-4dac-8bd3-4e19b1b08587 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:44.303200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3db2a778-0d7d-4dac-8bd3-4e19b1b08587 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:44.303969 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3db2a778-0d7d-4dac-8bd3-4e19b1b08587 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:44.304398 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3db2a778-0d7d-4dac-8bd3-4e19b1b08587 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:44.353123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f87dc9bb-f245-4305-abbf-0417620f227e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:44.354029 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f87dc9bb-f245-4305-abbf-0417620f227e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:44.361335 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f87dc9bb-f245-4305-abbf-0417620f227e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:44.361901 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f87dc9bb-f245-4305-abbf-0417620f227e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:44.362770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f87dc9bb-f245-4305-abbf-0417620f227e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:44.363424 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f87dc9bb-f245-4305-abbf-0417620f227e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:44.396491 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f87dc9bb-f245-4305-abbf-0417620f227e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:44.397023 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f87dc9bb-f245-4305-abbf-0417620f227e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:44.397570 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-f87dc9bb-f245-4305-abbf-0417620f227e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lazy-loading 'fault' on Instance uuid 161bde4a-7b5a-4c34-9cad-add4855d9d1f {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:39:44.439836 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3db2a778-0d7d-4dac-8bd3-4e19b1b08587 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:44.443100 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f87dc9bb-f245-4305-abbf-0417620f227e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:44.448315 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3db2a778-0d7d-4dac-8bd3-4e19b1b08587 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1654 microversion: 2.1 time: 0.156528 May 28 03:39:44.448880 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 258/531] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:44 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1654 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:44.453866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f87dc9bb-f245-4305-abbf-0417620f227e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1331 microversion: 2.1 time: 0.104913 May 28 03:39:44.454514 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 274/532] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:44 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1331 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:39:44.465075 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6633906c-2036-4dcf-97c2-2a6f84316fd3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:44.471164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6633906c-2036-4dcf-97c2-2a6f84316fd3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:44.471618 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6633906c-2036-4dcf-97c2-2a6f84316fd3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:44.493841 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6633906c-2036-4dcf-97c2-2a6f84316fd3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Going to try to start instance {{(pid=124605) start /opt/stack/nova/nova/compute/api.py:2730}} May 28 03:39:44.646556 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6633906c-2036-4dcf-97c2-2a6f84316fd3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "POST /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/action" status: 202 len: 0 microversion: 2.1 time: 0.185078 May 28 03:39:44.647163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 259/533] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:39:44 2022] POST /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/action => generated 0 bytes in 186 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:39:44.658321 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1fbbe1a0-beb1-450a-8259-9770d0da7a6e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:44.659004 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1fbbe1a0-beb1-450a-8259-9770d0da7a6e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:44.665722 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1fbbe1a0-beb1-450a-8259-9770d0da7a6e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:44.666165 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1fbbe1a0-beb1-450a-8259-9770d0da7a6e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:44.666938 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1fbbe1a0-beb1-450a-8259-9770d0da7a6e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:44.667368 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1fbbe1a0-beb1-450a-8259-9770d0da7a6e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:44.806954 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1fbbe1a0-beb1-450a-8259-9770d0da7a6e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:44.814392 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1fbbe1a0-beb1-450a-8259-9770d0da7a6e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1663 microversion: 2.1 time: 0.159906 May 28 03:39:44.814967 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 275/534] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:44 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1663 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:45.472141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-976466f4-9fba-4efe-bae1-1fb57f74968e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:45.472780 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-976466f4-9fba-4efe-bae1-1fb57f74968e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:45.479407 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-976466f4-9fba-4efe-bae1-1fb57f74968e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:45.479844 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-976466f4-9fba-4efe-bae1-1fb57f74968e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:45.480595 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-976466f4-9fba-4efe-bae1-1fb57f74968e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:45.481024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-976466f4-9fba-4efe-bae1-1fb57f74968e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:45.511337 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-976466f4-9fba-4efe-bae1-1fb57f74968e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:45.511775 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-976466f4-9fba-4efe-bae1-1fb57f74968e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:45.512213 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-976466f4-9fba-4efe-bae1-1fb57f74968e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lazy-loading 'fault' on Instance uuid 161bde4a-7b5a-4c34-9cad-add4855d9d1f {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:39:45.555500 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-976466f4-9fba-4efe-bae1-1fb57f74968e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:45.563540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-976466f4-9fba-4efe-bae1-1fb57f74968e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1331 microversion: 2.1 time: 0.096189 May 28 03:39:45.564101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 260/535] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:45 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1331 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:45.831701 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-615cd137-b799-431c-9f4a-e53a667dd31b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:45.832339 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-615cd137-b799-431c-9f4a-e53a667dd31b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:45.839240 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-615cd137-b799-431c-9f4a-e53a667dd31b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:45.839678 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-615cd137-b799-431c-9f4a-e53a667dd31b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:45.840441 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-615cd137-b799-431c-9f4a-e53a667dd31b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:45.840872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-615cd137-b799-431c-9f4a-e53a667dd31b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:45.967436 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-615cd137-b799-431c-9f4a-e53a667dd31b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:45.974801 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-615cd137-b799-431c-9f4a-e53a667dd31b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1663 microversion: 2.1 time: 0.147083 May 28 03:39:45.975367 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 276/536] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:45 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1663 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:46.580281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-16102000-e445-4113-a017-6e6936f8f062 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:46.580925 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-16102000-e445-4113-a017-6e6936f8f062 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:46.587563 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16102000-e445-4113-a017-6e6936f8f062 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:46.588008 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16102000-e445-4113-a017-6e6936f8f062 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:46.588770 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16102000-e445-4113-a017-6e6936f8f062 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:46.589227 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16102000-e445-4113-a017-6e6936f8f062 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:46.619929 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16102000-e445-4113-a017-6e6936f8f062 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:46.620371 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-16102000-e445-4113-a017-6e6936f8f062 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:46.620818 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-16102000-e445-4113-a017-6e6936f8f062 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lazy-loading 'fault' on Instance uuid 161bde4a-7b5a-4c34-9cad-add4855d9d1f {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:39:46.660775 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-16102000-e445-4113-a017-6e6936f8f062 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:46.667992 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-16102000-e445-4113-a017-6e6936f8f062 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 200 len: 1331 microversion: 2.1 time: 0.091609 May 28 03:39:46.668548 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 261/537] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:46 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 1331 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:46.992820 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1d5d47e0-64a2-46ab-a37b-49cc8b3f53f8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:46.994090 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1d5d47e0-64a2-46ab-a37b-49cc8b3f53f8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:47.000554 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d5d47e0-64a2-46ab-a37b-49cc8b3f53f8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:47.001002 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d5d47e0-64a2-46ab-a37b-49cc8b3f53f8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:47.001794 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d5d47e0-64a2-46ab-a37b-49cc8b3f53f8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:47.002228 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d5d47e0-64a2-46ab-a37b-49cc8b3f53f8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:47.136584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1d5d47e0-64a2-46ab-a37b-49cc8b3f53f8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:47.144413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1d5d47e0-64a2-46ab-a37b-49cc8b3f53f8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1663 microversion: 2.1 time: 0.155602 May 28 03:39:47.145131 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 277/538] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:46 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1663 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:47.684896 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ce389cf6-64fc-4817-a6b3-0756a046a6fa tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:47.685470 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ce389cf6-64fc-4817-a6b3-0756a046a6fa tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:47.691949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ce389cf6-64fc-4817-a6b3-0756a046a6fa tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:47.692399 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ce389cf6-64fc-4817-a6b3-0756a046a6fa tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:47.693190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ce389cf6-64fc-4817-a6b3-0756a046a6fa tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:47.693633 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ce389cf6-64fc-4817-a6b3-0756a046a6fa tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:47.709290 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-ce389cf6-64fc-4817-a6b3-0756a046a6fa tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ce389cf6-64fc-4817-a6b3-0756a046a6fa tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f HTTP/1.1 May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:39:47.710899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:39:47.712949 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '161bde4a-7b5a-4c34-9cad-add4855d9d1f'} May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:39:47.713875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:39:47.715251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:39:47.716572 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ce389cf6-64fc-4817-a6b3-0756a046a6fa tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:39:47.716572 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ce389cf6-64fc-4817-a6b3-0756a046a6fa tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 404 len: 111 microversion: 2.1 time: 0.031303 May 28 03:39:47.716572 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 262/539] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:47 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:39:47.721944 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4a2f87c9-c89a-491b-9c3a-e0f543cc6a0f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '.version_select at 0x7feb1f5ed430>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:47.804835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4a2f87c9-c89a-491b-9c3a-e0f543cc6a0f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2087622025" status: 202 len: 0 microversion: 2.1 time: 0.086798 May 28 03:39:47.805432 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 278/540] 10.222.0.41 () {58 vars in 1220 bytes} [Sat May 28 03:39:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2087622025 => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 28 03:39:48.161671 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-84ed06fb-04d1-4d1d-b4e0-77c3e14a2b19 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:48.162311 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-84ed06fb-04d1-4d1d-b4e0-77c3e14a2b19 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:48.170655 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-84ed06fb-04d1-4d1d-b4e0-77c3e14a2b19 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:48.171120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-84ed06fb-04d1-4d1d-b4e0-77c3e14a2b19 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:48.171890 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-84ed06fb-04d1-4d1d-b4e0-77c3e14a2b19 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:48.172328 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-84ed06fb-04d1-4d1d-b4e0-77c3e14a2b19 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:48.327435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-84ed06fb-04d1-4d1d-b4e0-77c3e14a2b19 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:48.335971 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-84ed06fb-04d1-4d1d-b4e0-77c3e14a2b19 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1663 microversion: 2.1 time: 0.177959 May 28 03:39:48.337304 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 263/541] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:48 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1663 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:48.403845 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b5be7cbf-696d-4406-b1c9-b85517b17110 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3e4c9aa3-01fe-4aaa-bdfd-4cf2851d934c"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:48.404906 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b5be7cbf-696d-4406-b1c9-b85517b17110 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:48.411915 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b5be7cbf-696d-4406-b1c9-b85517b17110 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:48.412373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b5be7cbf-696d-4406-b1c9-b85517b17110 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:48.413181 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b5be7cbf-696d-4406-b1c9-b85517b17110 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:48.413629 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b5be7cbf-696d-4406-b1c9-b85517b17110 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:48.586771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f182868e-7134-43a6-a41e-8184bb73c346 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: .version_select at 0x7feb1fcba790>, body: {"keypair": {"name": "tempest-keypair-978379138", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBF6u3qYOm9KJ5ijAfOaDPjAFBMkoniJP+4hPEQKdwCZjp6Wp/l825mipsSHfWABTOLxHsZynw6jz+44RPhKnvldU01SdaF91rcRbjVC3AtY2s/d0PJ++uYpNaxHgSyEg1A=="}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:48.600549 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-f182868e-7134-43a6-a41e-8184bb73c346 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:39:48.605343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-f182868e-7134-43a6-a41e-8184bb73c346 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for user 9d420e4b554e4deab21b89094e0547ca and project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:39:48.702395 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f182868e-7134-43a6-a41e-8184bb73c346 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.119112 May 28 03:39:48.703700 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 264/542] 10.222.0.41 () {60 vars in 1158 bytes} [Sat May 28 03:39:48 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:39:49.355232 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-665d555d-55af-462d-866e-071270113ce2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:49.355232 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-665d555d-55af-462d-866e-071270113ce2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:49.360515 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-665d555d-55af-462d-866e-071270113ce2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:49.361043 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-665d555d-55af-462d-866e-071270113ce2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:49.362013 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-665d555d-55af-462d-866e-071270113ce2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:49.362453 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-665d555d-55af-462d-866e-071270113ce2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:49.386444 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b5be7cbf-696d-4406-b1c9-b85517b17110 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "POST /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.986763 May 28 03:39:49.387168 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 279/543] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:39:48 2022] POST /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments => generated 194 bytes in 988 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:39:49.523628 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-665d555d-55af-462d-866e-071270113ce2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:49.534655 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-665d555d-55af-462d-866e-071270113ce2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1667 microversion: 2.1 time: 0.184883 May 28 03:39:49.535665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 265/544] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:49 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1667 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:39:51.295719 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1845376074", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ee7025a6-8632-48bc-9a4d-e0b6d8e98a79"}], "security_groups": [{"name": "tempest-securitygroup--2028642488"}], "key_name": "tempest-keypair-978379138", "user_data": "}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:51.398358 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] validate_networks() for [('ee7025a6-8632-48bc-9a4d-e0b6d8e98a79', None, None, None, None, None)] {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:39:51.763680 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:39:51.764111 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:39:51.764592 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:39:51.765004 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:39:51.765460 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:39:51.774337 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:51.775299 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:51.776111 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:51.776549 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:51.793581 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:39:51.797450 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for user 9d420e4b554e4deab21b89094e0547ca and project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:39:51.808234 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:39:51.810587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:39:51.920013 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e3aa76c0-3b9f-4f90-a3b2-c2e1b5118ce8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.629505 May 28 03:39:51.920781 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 280/545] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:39:51 2022] POST /compute/v2.1/servers => generated 402 bytes in 631 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:39:51.930486 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-deca2cbf-012f-40a6-b52b-98a4ca0d99ef tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:51.931641 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-deca2cbf-012f-40a6-b52b-98a4ca0d99ef tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:51.994346 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-deca2cbf-012f-40a6-b52b-98a4ca0d99ef tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:52.001150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-deca2cbf-012f-40a6-b52b-98a4ca0d99ef tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1241 microversion: 2.1 time: 0.074409 May 28 03:39:52.001721 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 266/546] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:51 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1241 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:53.017760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0fdb976f-92c1-411a-b21f-bc4d19f5462c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:53.018358 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0fdb976f-92c1-411a-b21f-bc4d19f5462c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:53.025348 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0fdb976f-92c1-411a-b21f-bc4d19f5462c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:53.025798 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0fdb976f-92c1-411a-b21f-bc4d19f5462c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:53.026596 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0fdb976f-92c1-411a-b21f-bc4d19f5462c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:53.027028 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0fdb976f-92c1-411a-b21f-bc4d19f5462c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:53.086687 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0fdb976f-92c1-411a-b21f-bc4d19f5462c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:53.090493 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:53.094686 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0fdb976f-92c1-411a-b21f-bc4d19f5462c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1245 microversion: 2.1 time: 0.081428 May 28 03:39:53.095250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 281/547] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:53 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1245 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:53.249491 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e079755b-026d-4af8-9f30-1988e43ca1a4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:53.250024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e079755b-026d-4af8-9f30-1988e43ca1a4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:53.252544 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=89,nova_api:DELETE=1,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:53.257348 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e079755b-026d-4af8-9f30-1988e43ca1a4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:53.257794 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e079755b-026d-4af8-9f30-1988e43ca1a4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:53.258643 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e079755b-026d-4af8-9f30-1988e43ca1a4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:53.259085 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e079755b-026d-4af8-9f30-1988e43ca1a4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:53.290146 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e079755b-026d-4af8-9f30-1988e43ca1a4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments/13c9b26d-363a-4b6b-9c6e-9f00f1b68b50" status: 200 len: 194 microversion: 2.20 time: 0.044602 May 28 03:39:53.291320 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 267/548] 10.222.0.41 () {60 vars in 1453 bytes} [Sat May 28 03:39:53 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments/13c9b26d-363a-4b6b-9c6e-9f00f1b68b50 => generated 194 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:39:53.370372 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e6d7ec1e-f250-4aa4-9a5c-8a66614486af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:53.370930 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e6d7ec1e-f250-4aa4-9a5c-8a66614486af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:53.377598 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e6d7ec1e-f250-4aa4-9a5c-8a66614486af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:53.381288 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e6d7ec1e-f250-4aa4-9a5c-8a66614486af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:53.381288 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e6d7ec1e-f250-4aa4-9a5c-8a66614486af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:53.381288 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e6d7ec1e-f250-4aa4-9a5c-8a66614486af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:53.461424 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-e6d7ec1e-f250-4aa4-9a5c-8a66614486af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lazy-loading 'services' on Instance uuid 35a9c645-590f-49e1-ab03-af4167f10547 {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:39:53.642464 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e6d7ec1e-f250-4aa4-9a5c-8a66614486af tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "DELETE /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments/13c9b26d-363a-4b6b-9c6e-9f00f1b68b50" status: 202 len: 0 microversion: 2.20 time: 0.276902 May 28 03:39:53.643076 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 282/549] 10.222.0.41 () {60 vars in 1456 bytes} [Sat May 28 03:39:53 2022] DELETE /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments/13c9b26d-363a-4b6b-9c6e-9f00f1b68b50 => generated 0 bytes in 279 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 28 03:39:53.651990 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2e6f91e9-6b40-49d2-b3ee-6959746f39aa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:53.652970 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2e6f91e9-6b40-49d2-b3ee-6959746f39aa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:53.662029 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2e6f91e9-6b40-49d2-b3ee-6959746f39aa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:53.662029 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2e6f91e9-6b40-49d2-b3ee-6959746f39aa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:53.662029 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2e6f91e9-6b40-49d2-b3ee-6959746f39aa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:53.662029 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2e6f91e9-6b40-49d2-b3ee-6959746f39aa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:53.678356 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:53.691858 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2e6f91e9-6b40-49d2-b3ee-6959746f39aa tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044462 May 28 03:39:53.692418 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 268/550] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:39:53 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:39:54.111708 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1310055094", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "networks": [{"uuid": "c66573d2-3992-4577-bc6d-7740bb5602ab"}]}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:54.114343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f31a1fe7-eac1-4b0b-a187-55e437675106 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:54.114968 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f31a1fe7-eac1-4b0b-a187-55e437675106 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:54.124285 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f31a1fe7-eac1-4b0b-a187-55e437675106 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:54.124737 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f31a1fe7-eac1-4b0b-a187-55e437675106 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:54.125540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f31a1fe7-eac1-4b0b-a187-55e437675106 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:54.125975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f31a1fe7-eac1-4b0b-a187-55e437675106 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:54.168341 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] validate_networks() for [('c66573d2-3992-4577-bc6d-7740bb5602ab', None, None, None, None, None)] {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:39:54.194358 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f31a1fe7-eac1-4b0b-a187-55e437675106 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:54.202257 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f31a1fe7-eac1-4b0b-a187-55e437675106 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1293 microversion: 2.1 time: 0.094559 May 28 03:39:54.203137 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 269/551] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:54 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1293 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:54.525412 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:39:54.529577 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:39:54.529577 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:39:54.529577 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:39:54.529577 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:39:54.538061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:54.538771 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:54.540343 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:54.541058 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:54.554052 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:54.560235 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Getting quotas for project e302a2cd63ba46d7a6cef952ddb89242. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:39:54.564345 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Getting quotas for user af5fa544bc6440c9830d25e4ad4d5987 and project e302a2cd63ba46d7a6cef952ddb89242. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:39:54.576246 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:39:54.578651 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:39:54.671584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-34d091ec-3819-49fa-8838-2cfdb583d71c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.564327 May 28 03:39:54.672672 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 283/552] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:39:54 2022] POST /compute/v2.1/servers => generated 380 bytes in 567 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:39:54.684579 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-76b6d6fd-94a9-4479-8f7f-b28d8bc10a17 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:54.685924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-76b6d6fd-94a9-4479-8f7f-b28d8bc10a17 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:54.704345 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8aad3520-4d48-4d35-a8de-0220daee3fea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:54.705443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8aad3520-4d48-4d35-a8de-0220daee3fea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:54.724112 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8aad3520-4d48-4d35-a8de-0220daee3fea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:54.724571 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8aad3520-4d48-4d35-a8de-0220daee3fea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:54.725405 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8aad3520-4d48-4d35-a8de-0220daee3fea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:54.725844 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8aad3520-4d48-4d35-a8de-0220daee3fea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:54.785062 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8aad3520-4d48-4d35-a8de-0220daee3fea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.084462 May 28 03:39:54.785969 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 284/553] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:39:54 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:39:54.892688 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-76b6d6fd-94a9-4479-8f7f-b28d8bc10a17 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:54.899263 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-76b6d6fd-94a9-4479-8f7f-b28d8bc10a17 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1219 microversion: 2.1 time: 0.220038 May 28 03:39:54.899859 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 270/554] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:54 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1219 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:55.225291 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d6efface-baf3-415f-a934-022a3c718503 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:55.226549 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d6efface-baf3-415f-a934-022a3c718503 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:55.234124 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d6efface-baf3-415f-a934-022a3c718503 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:55.235232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d6efface-baf3-415f-a934-022a3c718503 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:55.236197 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d6efface-baf3-415f-a934-022a3c718503 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:55.236838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d6efface-baf3-415f-a934-022a3c718503 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:55.310919 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d6efface-baf3-415f-a934-022a3c718503 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:55.318843 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d6efface-baf3-415f-a934-022a3c718503 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1299 microversion: 2.1 time: 0.099423 May 28 03:39:55.319338 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 285/555] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:55 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1299 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:55.796615 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ec0dc00b-870f-41a2-af0b-21ce054ec9ce tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:55.797587 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ec0dc00b-870f-41a2-af0b-21ce054ec9ce tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:55.804950 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ec0dc00b-870f-41a2-af0b-21ce054ec9ce tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:55.805629 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ec0dc00b-870f-41a2-af0b-21ce054ec9ce tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:55.806427 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ec0dc00b-870f-41a2-af0b-21ce054ec9ce tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:55.807057 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ec0dc00b-870f-41a2-af0b-21ce054ec9ce tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:55.837806 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ec0dc00b-870f-41a2-af0b-21ce054ec9ce tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044533 May 28 03:39:55.838293 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 271/556] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:39:55 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:39:55.915934 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dd344737-1b50-4d34-bfe5-fb586e6f89d4 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:55.916544 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dd344737-1b50-4d34-bfe5-fb586e6f89d4 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:55.923689 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd344737-1b50-4d34-bfe5-fb586e6f89d4 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:55.924136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd344737-1b50-4d34-bfe5-fb586e6f89d4 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:55.928007 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd344737-1b50-4d34-bfe5-fb586e6f89d4 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:55.928007 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd344737-1b50-4d34-bfe5-fb586e6f89d4 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:55.988714 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dd344737-1b50-4d34-bfe5-fb586e6f89d4 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:55.998368 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-dd344737-1b50-4d34-bfe5-fb586e6f89d4 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1223 microversion: 2.1 time: 0.086343 May 28 03:39:55.998946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 286/557] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:55 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1223 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:56.339938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c29be36e-3fc8-40be-95ce-9d5666fa0fa7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:56.340649 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c29be36e-3fc8-40be-95ce-9d5666fa0fa7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:56.347092 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c29be36e-3fc8-40be-95ce-9d5666fa0fa7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:56.347537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c29be36e-3fc8-40be-95ce-9d5666fa0fa7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:56.348314 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c29be36e-3fc8-40be-95ce-9d5666fa0fa7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:56.348758 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c29be36e-3fc8-40be-95ce-9d5666fa0fa7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:56.544795 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c29be36e-3fc8-40be-95ce-9d5666fa0fa7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:56.552298 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c29be36e-3fc8-40be-95ce-9d5666fa0fa7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.216537 May 28 03:39:56.552867 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 272/558] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:56 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:56.697279 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-ae82896b-d275-4e7b-ad3a-fc65b5d87cdf req-d6299523-60c5-4fa4-961a-34a3106014b1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c2d3a613-c1f1-49d5-a707-784bdc430953", "tag": "f1321a9e-949d-4d52-abcc-e48ee7436a3a"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:39:56.706975 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-ae82896b-d275-4e7b-ad3a-fc65b5d87cdf req-d6299523-60c5-4fa4-961a-34a3106014b1 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:56.707601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-ae82896b-d275-4e7b-ad3a-fc65b5d87cdf req-d6299523-60c5-4fa4-961a-34a3106014b1 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:56.734627 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-ae82896b-d275-4e7b-ad3a-fc65b5d87cdf req-d6299523-60c5-4fa4-961a-34a3106014b1 service nova] Creating event network-changed:f1321a9e-949d-4d52-abcc-e48ee7436a3a for instance c2d3a613-c1f1-49d5-a707-784bdc430953 on hv-cinder-84336 May 28 03:39:56.741132 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-ae82896b-d275-4e7b-ad3a-fc65b5d87cdf req-d6299523-60c5-4fa4-961a-34a3106014b1 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047562 May 28 03:39:56.748213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 287/559] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:39:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:39:56.849868 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2affc692-049d-4151-bb41-4ab46fd139e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:56.850889 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2affc692-049d-4151-bb41-4ab46fd139e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:56.857718 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2affc692-049d-4151-bb41-4ab46fd139e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:56.858411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2affc692-049d-4151-bb41-4ab46fd139e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:56.859902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2affc692-049d-4151-bb41-4ab46fd139e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:56.860537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2affc692-049d-4151-bb41-4ab46fd139e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:56.891185 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2affc692-049d-4151-bb41-4ab46fd139e7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045481 May 28 03:39:56.895894 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 273/560] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:39:56 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:39:57.016866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e8d336da-ea3b-441c-a118-5d21447bb51c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:57.017603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e8d336da-ea3b-441c-a118-5d21447bb51c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:57.024009 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e8d336da-ea3b-441c-a118-5d21447bb51c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:57.024492 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e8d336da-ea3b-441c-a118-5d21447bb51c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:57.026246 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e8d336da-ea3b-441c-a118-5d21447bb51c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:57.026679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e8d336da-ea3b-441c-a118-5d21447bb51c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:57.091448 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e8d336da-ea3b-441c-a118-5d21447bb51c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:57.098824 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e8d336da-ea3b-441c-a118-5d21447bb51c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.086410 May 28 03:39:57.099581 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 288/561] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:57 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:57.570087 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-092fc3a4-b130-4fa4-9a70-379831105d0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:57.570703 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-092fc3a4-b130-4fa4-9a70-379831105d0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:57.576633 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-092fc3a4-b130-4fa4-9a70-379831105d0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:57.577082 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-092fc3a4-b130-4fa4-9a70-379831105d0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:57.577881 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-092fc3a4-b130-4fa4-9a70-379831105d0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:57.578323 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-092fc3a4-b130-4fa4-9a70-379831105d0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:57.710143 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-092fc3a4-b130-4fa4-9a70-379831105d0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:57.718158 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-092fc3a4-b130-4fa4-9a70-379831105d0b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.152007 May 28 03:39:57.718716 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 274/562] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:57 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:57.903274 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9ac5ae42-5557-42ef-ad8e-c8a45d1f3186 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:57.904209 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9ac5ae42-5557-42ef-ad8e-c8a45d1f3186 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:57.910881 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9ac5ae42-5557-42ef-ad8e-c8a45d1f3186 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:57.911354 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9ac5ae42-5557-42ef-ad8e-c8a45d1f3186 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:57.912135 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9ac5ae42-5557-42ef-ad8e-c8a45d1f3186 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:57.912581 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9ac5ae42-5557-42ef-ad8e-c8a45d1f3186 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:57.941944 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9ac5ae42-5557-42ef-ad8e-c8a45d1f3186 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043227 May 28 03:39:57.942500 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 289/563] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:39:57 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:39:58.115964 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b4826623-bdf4-42a7-8a30-ef156bb1697a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:58.116569 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-b4826623-bdf4-42a7-8a30-ef156bb1697a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:58.123177 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b4826623-bdf4-42a7-8a30-ef156bb1697a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:58.123617 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b4826623-bdf4-42a7-8a30-ef156bb1697a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:58.124418 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b4826623-bdf4-42a7-8a30-ef156bb1697a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:58.124872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b4826623-bdf4-42a7-8a30-ef156bb1697a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:58.192604 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-b4826623-bdf4-42a7-8a30-ef156bb1697a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:58.200170 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-b4826623-bdf4-42a7-8a30-ef156bb1697a tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.088375 May 28 03:39:58.200733 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 275/564] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:58 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:58.735583 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e5412421-b087-439c-bba3-6ca7ea1ac919 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:58.736192 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e5412421-b087-439c-bba3-6ca7ea1ac919 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:58.742566 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e5412421-b087-439c-bba3-6ca7ea1ac919 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:58.748764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e5412421-b087-439c-bba3-6ca7ea1ac919 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:58.748764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e5412421-b087-439c-bba3-6ca7ea1ac919 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:58.748764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e5412421-b087-439c-bba3-6ca7ea1ac919 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:58.871885 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e5412421-b087-439c-bba3-6ca7ea1ac919 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:58.879187 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e5412421-b087-439c-bba3-6ca7ea1ac919 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.147539 May 28 03:39:58.879747 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 290/565] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:58 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:58.953299 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-321ea8b7-919e-41ee-b817-d822d45e1c7a tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:58.954250 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-321ea8b7-919e-41ee-b817-d822d45e1c7a tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:58.960447 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-321ea8b7-919e-41ee-b817-d822d45e1c7a tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:58.960898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-321ea8b7-919e-41ee-b817-d822d45e1c7a tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:58.961685 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-321ea8b7-919e-41ee-b817-d822d45e1c7a tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:58.962117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-321ea8b7-919e-41ee-b817-d822d45e1c7a tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:58.992003 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-321ea8b7-919e-41ee-b817-d822d45e1c7a tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042712 May 28 03:39:58.993428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 276/566] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:39:58 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:39:59.217946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4183d355-659a-4e2f-a6fa-88644760f9bb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:59.218476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4183d355-659a-4e2f-a6fa-88644760f9bb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:59.224915 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4183d355-659a-4e2f-a6fa-88644760f9bb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:59.225393 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4183d355-659a-4e2f-a6fa-88644760f9bb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:59.226159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4183d355-659a-4e2f-a6fa-88644760f9bb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:59.226600 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4183d355-659a-4e2f-a6fa-88644760f9bb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:59.286402 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4183d355-659a-4e2f-a6fa-88644760f9bb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:39:59.293509 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4183d355-659a-4e2f-a6fa-88644760f9bb tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.080366 May 28 03:39:59.294072 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 291/567] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:59 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:39:59.895817 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-86b0b7db-47d2-4b73-b032-ee66b55b5f19 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:39:59.896429 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-86b0b7db-47d2-4b73-b032-ee66b55b5f19 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:39:59.903598 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-86b0b7db-47d2-4b73-b032-ee66b55b5f19 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:59.904049 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-86b0b7db-47d2-4b73-b032-ee66b55b5f19 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:39:59.904818 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-86b0b7db-47d2-4b73-b032-ee66b55b5f19 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:39:59.905289 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-86b0b7db-47d2-4b73-b032-ee66b55b5f19 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:00.007013 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1bb067f4-222f-47d7-bb20-c9775fc4a662 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:00.007935 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1bb067f4-222f-47d7-bb20-c9775fc4a662 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:00.013787 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1bb067f4-222f-47d7-bb20-c9775fc4a662 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:00.014237 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1bb067f4-222f-47d7-bb20-c9775fc4a662 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:00.015006 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1bb067f4-222f-47d7-bb20-c9775fc4a662 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:00.015433 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1bb067f4-222f-47d7-bb20-c9775fc4a662 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:00.043155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-86b0b7db-47d2-4b73-b032-ee66b55b5f19 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:00.045324 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1bb067f4-222f-47d7-bb20-c9775fc4a662 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042217 May 28 03:40:00.046111 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 292/568] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:00 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:40:00.054880 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-86b0b7db-47d2-4b73-b032-ee66b55b5f19 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.163096 May 28 03:40:00.055447 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 277/569] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:39:59 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:00.310736 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9e68c9b1-f658-485d-be5d-b91816419470 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:00.311350 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9e68c9b1-f658-485d-be5d-b91816419470 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:00.317871 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9e68c9b1-f658-485d-be5d-b91816419470 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:00.318320 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9e68c9b1-f658-485d-be5d-b91816419470 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:00.319096 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9e68c9b1-f658-485d-be5d-b91816419470 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:00.319537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9e68c9b1-f658-485d-be5d-b91816419470 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:00.386143 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9e68c9b1-f658-485d-be5d-b91816419470 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:00.393464 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9e68c9b1-f658-485d-be5d-b91816419470 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.087085 May 28 03:40:00.394033 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 293/570] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:00 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:01.056858 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3ebc50e7-f36c-45c9-a54e-f7dcff3a60ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:01.058874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3ebc50e7-f36c-45c9-a54e-f7dcff3a60ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:01.064443 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ebc50e7-f36c-45c9-a54e-f7dcff3a60ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:01.064894 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ebc50e7-f36c-45c9-a54e-f7dcff3a60ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:01.065695 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ebc50e7-f36c-45c9-a54e-f7dcff3a60ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:01.066128 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ebc50e7-f36c-45c9-a54e-f7dcff3a60ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:01.071510 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e2a9a853-104c-4bee-8560-cac91b8d7eb9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:01.072205 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e2a9a853-104c-4bee-8560-cac91b8d7eb9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:01.078297 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2a9a853-104c-4bee-8560-cac91b8d7eb9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:01.078824 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2a9a853-104c-4bee-8560-cac91b8d7eb9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:01.080095 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2a9a853-104c-4bee-8560-cac91b8d7eb9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:01.080095 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2a9a853-104c-4bee-8560-cac91b8d7eb9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:01.095836 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3ebc50e7-f36c-45c9-a54e-f7dcff3a60ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042831 May 28 03:40:01.096781 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 278/571] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:01 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:40:01.211296 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e2a9a853-104c-4bee-8560-cac91b8d7eb9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:01.218568 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e2a9a853-104c-4bee-8560-cac91b8d7eb9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.150878 May 28 03:40:01.219117 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 294/572] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:01 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:01.410409 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c703a90d-273f-46b0-a5e5-adeacd546375 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:01.410934 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c703a90d-273f-46b0-a5e5-adeacd546375 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:01.418701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c703a90d-273f-46b0-a5e5-adeacd546375 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:01.419461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c703a90d-273f-46b0-a5e5-adeacd546375 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:01.419941 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c703a90d-273f-46b0-a5e5-adeacd546375 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:01.420375 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c703a90d-273f-46b0-a5e5-adeacd546375 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:01.482133 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c703a90d-273f-46b0-a5e5-adeacd546375 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:01.489807 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c703a90d-273f-46b0-a5e5-adeacd546375 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.083343 May 28 03:40:01.490371 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 279/573] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:01 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:02.107744 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1b4ab6dd-9a8d-43d8-99c4-b8d3dcf53ef7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:02.108669 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1b4ab6dd-9a8d-43d8-99c4-b8d3dcf53ef7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:02.115271 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1b4ab6dd-9a8d-43d8-99c4-b8d3dcf53ef7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:02.116548 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1b4ab6dd-9a8d-43d8-99c4-b8d3dcf53ef7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:02.116741 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1b4ab6dd-9a8d-43d8-99c4-b8d3dcf53ef7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:02.117203 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1b4ab6dd-9a8d-43d8-99c4-b8d3dcf53ef7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:02.147414 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1b4ab6dd-9a8d-43d8-99c4-b8d3dcf53ef7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043677 May 28 03:40:02.152407 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 295/574] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:02 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:40:02.235461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-80729a2b-b3eb-4b27-b004-20c3eff6945f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:02.236066 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-80729a2b-b3eb-4b27-b004-20c3eff6945f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:02.242764 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-80729a2b-b3eb-4b27-b004-20c3eff6945f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:02.243208 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-80729a2b-b3eb-4b27-b004-20c3eff6945f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:02.243987 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-80729a2b-b3eb-4b27-b004-20c3eff6945f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:02.244943 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-80729a2b-b3eb-4b27-b004-20c3eff6945f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:02.379318 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-80729a2b-b3eb-4b27-b004-20c3eff6945f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:02.384438 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-80729a2b-b3eb-4b27-b004-20c3eff6945f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.152786 May 28 03:40:02.385024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 280/575] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:02 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:02.507399 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2de1d0f6-9251-442f-a963-24a990ae13b2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:02.513166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2de1d0f6-9251-442f-a963-24a990ae13b2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:02.514082 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2de1d0f6-9251-442f-a963-24a990ae13b2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:02.514534 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2de1d0f6-9251-442f-a963-24a990ae13b2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:02.515295 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2de1d0f6-9251-442f-a963-24a990ae13b2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:02.515769 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2de1d0f6-9251-442f-a963-24a990ae13b2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:02.576816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-2de1d0f6-9251-442f-a963-24a990ae13b2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:02.584065 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2de1d0f6-9251-442f-a963-24a990ae13b2 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.080540 May 28 03:40:02.584608 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 296/576] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:02 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:03.159702 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f3432504-cdb1-4958-9d00-70cb59e85e16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:03.160615 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f3432504-cdb1-4958-9d00-70cb59e85e16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:03.167199 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3432504-cdb1-4958-9d00-70cb59e85e16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:03.167663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3432504-cdb1-4958-9d00-70cb59e85e16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:03.168452 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3432504-cdb1-4958-9d00-70cb59e85e16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:03.168872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3432504-cdb1-4958-9d00-70cb59e85e16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:03.196594 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f3432504-cdb1-4958-9d00-70cb59e85e16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040807 May 28 03:40:03.197207 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 281/577] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:03 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:40:03.401775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f7abce60-75ff-4e46-8c49-9bc2fa360ad7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:03.402383 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f7abce60-75ff-4e46-8c49-9bc2fa360ad7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:03.408976 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f7abce60-75ff-4e46-8c49-9bc2fa360ad7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:03.409460 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f7abce60-75ff-4e46-8c49-9bc2fa360ad7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:03.410270 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f7abce60-75ff-4e46-8c49-9bc2fa360ad7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:03.410655 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f7abce60-75ff-4e46-8c49-9bc2fa360ad7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:03.538648 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f7abce60-75ff-4e46-8c49-9bc2fa360ad7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:03.546193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f7abce60-75ff-4e46-8c49-9bc2fa360ad7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.148787 May 28 03:40:03.546720 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 297/578] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:03 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:03.599653 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7c2cdcb2-23cf-4cd0-972a-0cc90958b7be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:03.600254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7c2cdcb2-23cf-4cd0-972a-0cc90958b7be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:03.606642 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7c2cdcb2-23cf-4cd0-972a-0cc90958b7be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:03.607079 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7c2cdcb2-23cf-4cd0-972a-0cc90958b7be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:03.607830 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7c2cdcb2-23cf-4cd0-972a-0cc90958b7be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:03.608258 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7c2cdcb2-23cf-4cd0-972a-0cc90958b7be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:03.673027 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7c2cdcb2-23cf-4cd0-972a-0cc90958b7be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:03.680521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7c2cdcb2-23cf-4cd0-972a-0cc90958b7be tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.084701 May 28 03:40:03.681129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 282/579] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:03 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:04.208392 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0a9b97b4-16fa-4015-8ad5-25b704be3f6e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:04.209256 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0a9b97b4-16fa-4015-8ad5-25b704be3f6e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:04.216056 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a9b97b4-16fa-4015-8ad5-25b704be3f6e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:04.216508 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a9b97b4-16fa-4015-8ad5-25b704be3f6e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:04.217306 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a9b97b4-16fa-4015-8ad5-25b704be3f6e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:04.217747 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a9b97b4-16fa-4015-8ad5-25b704be3f6e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:04.246585 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0a9b97b4-16fa-4015-8ad5-25b704be3f6e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042166 May 28 03:40:04.247179 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 298/580] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:04 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:40:04.563482 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5c8d9afa-0b7b-465c-95e9-8e421aad373e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:04.564085 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5c8d9afa-0b7b-465c-95e9-8e421aad373e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:04.570215 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5c8d9afa-0b7b-465c-95e9-8e421aad373e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:04.570654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5c8d9afa-0b7b-465c-95e9-8e421aad373e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:04.571404 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5c8d9afa-0b7b-465c-95e9-8e421aad373e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:04.571832 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5c8d9afa-0b7b-465c-95e9-8e421aad373e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:04.698300 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3d991b1d-ca08-4c5f-9de9-fe88dda9424f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:04.698979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3d991b1d-ca08-4c5f-9de9-fe88dda9424f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:04.699741 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5c8d9afa-0b7b-465c-95e9-8e421aad373e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:04.701506 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=4 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:04.702294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=12 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:04.706888 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3d991b1d-ca08-4c5f-9de9-fe88dda9424f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:04.707335 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3d991b1d-ca08-4c5f-9de9-fe88dda9424f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:04.707453 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5c8d9afa-0b7b-465c-95e9-8e421aad373e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.147700 May 28 03:40:04.708209 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 283/581] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:04 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:04.708332 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3d991b1d-ca08-4c5f-9de9-fe88dda9424f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:04.708538 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3d991b1d-ca08-4c5f-9de9-fe88dda9424f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:04.712570 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:04.769791 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3d991b1d-ca08-4c5f-9de9-fe88dda9424f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:04.777463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3d991b1d-ca08-4c5f-9de9-fe88dda9424f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.083936 May 28 03:40:04.778049 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 299/582] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:04 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:05.258283 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-be4fcf02-18a9-4d53-b8de-0701bbd5b7bb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:05.259195 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-be4fcf02-18a9-4d53-b8de-0701bbd5b7bb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:05.261615 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=8 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:05.266388 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-be4fcf02-18a9-4d53-b8de-0701bbd5b7bb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:05.266835 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-be4fcf02-18a9-4d53-b8de-0701bbd5b7bb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:05.267596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-be4fcf02-18a9-4d53-b8de-0701bbd5b7bb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:05.268026 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-be4fcf02-18a9-4d53-b8de-0701bbd5b7bb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:05.297609 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-be4fcf02-18a9-4d53-b8de-0701bbd5b7bb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043192 May 28 03:40:05.298169 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 284/583] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:05 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:40:05.723701 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-681174a0-a67e-415f-b014-ece97c28f6cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:05.724352 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-681174a0-a67e-415f-b014-ece97c28f6cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:05.732032 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-681174a0-a67e-415f-b014-ece97c28f6cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:05.732493 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-681174a0-a67e-415f-b014-ece97c28f6cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:05.733298 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-681174a0-a67e-415f-b014-ece97c28f6cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:05.733739 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-681174a0-a67e-415f-b014-ece97c28f6cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:05.794030 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9703a13f-17ec-4afc-8464-44828602350c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:05.794643 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9703a13f-17ec-4afc-8464-44828602350c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:05.801385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9703a13f-17ec-4afc-8464-44828602350c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:05.801827 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9703a13f-17ec-4afc-8464-44828602350c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:05.802582 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9703a13f-17ec-4afc-8464-44828602350c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:05.803007 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9703a13f-17ec-4afc-8464-44828602350c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:05.862788 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-681174a0-a67e-415f-b014-ece97c28f6cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:05.863674 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9703a13f-17ec-4afc-8464-44828602350c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:05.871175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-681174a0-a67e-415f-b014-ece97c28f6cf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.151467 May 28 03:40:05.871827 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 300/584] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:05 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:05.871947 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9703a13f-17ec-4afc-8464-44828602350c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.082182 May 28 03:40:05.872458 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 285/585] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:05 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:06.309707 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-370d86c4-fda4-4995-b21e-2759793c9f56 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:06.310632 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-370d86c4-fda4-4995-b21e-2759793c9f56 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:06.316516 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-370d86c4-fda4-4995-b21e-2759793c9f56 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:06.316961 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-370d86c4-fda4-4995-b21e-2759793c9f56 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:06.317762 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-370d86c4-fda4-4995-b21e-2759793c9f56 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:06.318193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-370d86c4-fda4-4995-b21e-2759793c9f56 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:06.346962 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-370d86c4-fda4-4995-b21e-2759793c9f56 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041625 May 28 03:40:06.347443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 301/586] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:06 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:40:06.888932 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ab187e25-1de4-4384-b55c-c92486cf760c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:06.889496 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-474dbc0b-c4bc-4c01-89a9-31257c5b7a56 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:06.889884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ab187e25-1de4-4384-b55c-c92486cf760c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:06.889989 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-474dbc0b-c4bc-4c01-89a9-31257c5b7a56 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:06.895982 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-474dbc0b-c4bc-4c01-89a9-31257c5b7a56 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:06.896129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ab187e25-1de4-4384-b55c-c92486cf760c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:06.896631 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-474dbc0b-c4bc-4c01-89a9-31257c5b7a56 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:06.896722 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ab187e25-1de4-4384-b55c-c92486cf760c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:06.897274 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-474dbc0b-c4bc-4c01-89a9-31257c5b7a56 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:06.897395 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ab187e25-1de4-4384-b55c-c92486cf760c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:06.897741 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ab187e25-1de4-4384-b55c-c92486cf760c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:06.897869 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-474dbc0b-c4bc-4c01-89a9-31257c5b7a56 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:06.902288 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=100 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:06.966222 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-474dbc0b-c4bc-4c01-89a9-31257c5b7a56 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:06.973998 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-474dbc0b-c4bc-4c01-89a9-31257c5b7a56 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.089484 May 28 03:40:06.974578 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 302/587] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:06 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:07.043546 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ab187e25-1de4-4384-b55c-c92486cf760c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:07.050990 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ab187e25-1de4-4384-b55c-c92486cf760c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.166658 May 28 03:40:07.051546 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 286/588] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:06 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:07.358881 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4151d736-ebfd-4bd5-abaa-3f41989271ac tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:07.359786 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4151d736-ebfd-4bd5-abaa-3f41989271ac tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:07.365757 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4151d736-ebfd-4bd5-abaa-3f41989271ac tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:07.366464 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4151d736-ebfd-4bd5-abaa-3f41989271ac tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:07.367209 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4151d736-ebfd-4bd5-abaa-3f41989271ac tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:07.367633 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4151d736-ebfd-4bd5-abaa-3f41989271ac tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:07.397075 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4151d736-ebfd-4bd5-abaa-3f41989271ac tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042016 May 28 03:40:07.397707 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 303/589] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:07 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:40:07.990787 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-669cd349-d233-4ece-b3a4-d560ef1c45f9 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:07.991666 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-669cd349-d233-4ece-b3a4-d560ef1c45f9 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:07.998461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-669cd349-d233-4ece-b3a4-d560ef1c45f9 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:07.998910 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-669cd349-d233-4ece-b3a4-d560ef1c45f9 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:07.999670 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-669cd349-d233-4ece-b3a4-d560ef1c45f9 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:08.000108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-669cd349-d233-4ece-b3a4-d560ef1c45f9 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:08.060630 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-669cd349-d233-4ece-b3a4-d560ef1c45f9 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:08.068184 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-669cd349-d233-4ece-b3a4-d560ef1c45f9 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.081379 May 28 03:40:08.068698 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-58299ad6-48dc-4831-a009-b2ba301c59e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:08.069164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 287/590] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:07 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:08.069468 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-58299ad6-48dc-4831-a009-b2ba301c59e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:08.075843 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-58299ad6-48dc-4831-a009-b2ba301c59e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:08.076285 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-58299ad6-48dc-4831-a009-b2ba301c59e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:08.077044 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-58299ad6-48dc-4831-a009-b2ba301c59e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:08.077500 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-58299ad6-48dc-4831-a009-b2ba301c59e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:08.204241 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-58299ad6-48dc-4831-a009-b2ba301c59e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:08.212738 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-58299ad6-48dc-4831-a009-b2ba301c59e9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.147739 May 28 03:40:08.213882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 304/591] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:08 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:08.407934 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-bda5b51e-b364-4fa7-8024-87d255cee705 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:08.408830 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-bda5b51e-b364-4fa7-8024-87d255cee705 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:08.414846 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bda5b51e-b364-4fa7-8024-87d255cee705 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:08.423189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bda5b51e-b364-4fa7-8024-87d255cee705 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:08.423189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bda5b51e-b364-4fa7-8024-87d255cee705 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:08.423189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bda5b51e-b364-4fa7-8024-87d255cee705 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:08.445347 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-bda5b51e-b364-4fa7-8024-87d255cee705 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041353 May 28 03:40:08.445939 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 288/592] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:08 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:40:09.084202 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8ababc7d-9f57-4b06-83d6-b46623936d7c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:09.089080 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8ababc7d-9f57-4b06-83d6-b46623936d7c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:09.091527 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8ababc7d-9f57-4b06-83d6-b46623936d7c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:09.091980 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8ababc7d-9f57-4b06-83d6-b46623936d7c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:09.092848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8ababc7d-9f57-4b06-83d6-b46623936d7c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:09.093319 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8ababc7d-9f57-4b06-83d6-b46623936d7c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:09.154062 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8ababc7d-9f57-4b06-83d6-b46623936d7c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:09.161305 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8ababc7d-9f57-4b06-83d6-b46623936d7c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1271 microversion: 2.1 time: 0.080869 May 28 03:40:09.163586 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 305/593] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:09 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1271 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:09.230203 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ff83b460-a1cd-4ea2-873a-b1fa097ca846 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:09.230961 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ff83b460-a1cd-4ea2-873a-b1fa097ca846 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:09.237672 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ff83b460-a1cd-4ea2-873a-b1fa097ca846 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:09.238120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ff83b460-a1cd-4ea2-873a-b1fa097ca846 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:09.238892 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ff83b460-a1cd-4ea2-873a-b1fa097ca846 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:09.239327 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ff83b460-a1cd-4ea2-873a-b1fa097ca846 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:09.385902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ff83b460-a1cd-4ea2-873a-b1fa097ca846 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:09.393325 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ff83b460-a1cd-4ea2-873a-b1fa097ca846 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.167053 May 28 03:40:09.393888 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 289/594] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:09 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:09.457649 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-383097f6-bd6f-4c88-827b-b20fdb83fef1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:09.458502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-383097f6-bd6f-4c88-827b-b20fdb83fef1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:09.465174 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-383097f6-bd6f-4c88-827b-b20fdb83fef1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:09.465629 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-383097f6-bd6f-4c88-827b-b20fdb83fef1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:09.466398 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-383097f6-bd6f-4c88-827b-b20fdb83fef1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:09.466831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-383097f6-bd6f-4c88-827b-b20fdb83fef1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:09.495140 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-383097f6-bd6f-4c88-827b-b20fdb83fef1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041455 May 28 03:40:09.495863 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 306/595] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:09 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:40:10.179376 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a33742e8-7d89-4871-896a-934e2b3587b0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:10.179991 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a33742e8-7d89-4871-896a-934e2b3587b0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:10.186537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a33742e8-7d89-4871-896a-934e2b3587b0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:10.186998 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a33742e8-7d89-4871-896a-934e2b3587b0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:10.187760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a33742e8-7d89-4871-896a-934e2b3587b0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:10.188190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a33742e8-7d89-4871-896a-934e2b3587b0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:10.307492 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a33742e8-7d89-4871-896a-934e2b3587b0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:10.315097 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a33742e8-7d89-4871-896a-934e2b3587b0 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1289 microversion: 2.1 time: 0.139521 May 28 03:40:10.315904 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 290/596] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:10 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1289 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:10.410664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-84463355-3493-434a-9067-d03284d910d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:10.411289 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-84463355-3493-434a-9067-d03284d910d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:10.418061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-84463355-3493-434a-9067-d03284d910d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:10.418572 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-84463355-3493-434a-9067-d03284d910d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:10.419345 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-84463355-3493-434a-9067-d03284d910d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:10.419783 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-84463355-3493-434a-9067-d03284d910d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:10.515998 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9080c38b-2dc4-4d09-ac21-4c5ae2d1b089 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:10.516950 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9080c38b-2dc4-4d09-ac21-4c5ae2d1b089 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:10.523749 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9080c38b-2dc4-4d09-ac21-4c5ae2d1b089 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:10.524192 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9080c38b-2dc4-4d09-ac21-4c5ae2d1b089 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:10.524952 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9080c38b-2dc4-4d09-ac21-4c5ae2d1b089 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:10.525412 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9080c38b-2dc4-4d09-ac21-4c5ae2d1b089 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:10.569554 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-84463355-3493-434a-9067-d03284d910d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:10.579472 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9080c38b-2dc4-4d09-ac21-4c5ae2d1b089 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.076842 May 28 03:40:10.580043 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-84463355-3493-434a-9067-d03284d910d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.172873 May 28 03:40:10.580846 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 307/597] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:10 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:10.583597 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 291/598] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:10 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:40:10.696509 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-7adb65ce-0d24-4f9e-a5d1-a89eeed3a4ea tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:10.697489 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-7adb65ce-0d24-4f9e-a5d1-a89eeed3a4ea tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:10.703452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7adb65ce-0d24-4f9e-a5d1-a89eeed3a4ea tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:10.703904 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7adb65ce-0d24-4f9e-a5d1-a89eeed3a4ea tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:10.704675 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7adb65ce-0d24-4f9e-a5d1-a89eeed3a4ea tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:10.705123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7adb65ce-0d24-4f9e-a5d1-a89eeed3a4ea tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:10.729135 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-7adb65ce-0d24-4f9e-a5d1-a89eeed3a4ea tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lazy-loading 'flavor' on Instance uuid 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:40:11.057228 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-7adb65ce-0d24-4f9e-a5d1-a89eeed3a4ea tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "POST /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/action" status: 200 len: 32 microversion: 2.1 time: 0.363874 May 28 03:40:11.062839 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 308/599] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:40:10 2022] POST /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/action => generated 32 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 28 03:40:11.067209 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-eb6db9be-135e-4457-a7f5-e8faa6013686 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:11.067824 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-eb6db9be-135e-4457-a7f5-e8faa6013686 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:11.074634 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eb6db9be-135e-4457-a7f5-e8faa6013686 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:11.075076 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eb6db9be-135e-4457-a7f5-e8faa6013686 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:11.075851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eb6db9be-135e-4457-a7f5-e8faa6013686 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:11.076289 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eb6db9be-135e-4457-a7f5-e8faa6013686 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:11.206782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-eb6db9be-135e-4457-a7f5-e8faa6013686 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:11.215290 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-eb6db9be-135e-4457-a7f5-e8faa6013686 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.151902 May 28 03:40:11.215848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 292/600] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:11 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:11.333281 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6f49214a-8a2d-43ee-bd6e-ea5a3592dfe6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:11.333978 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-6f49214a-8a2d-43ee-bd6e-ea5a3592dfe6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:11.340592 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6f49214a-8a2d-43ee-bd6e-ea5a3592dfe6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:11.341038 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6f49214a-8a2d-43ee-bd6e-ea5a3592dfe6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:11.341826 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6f49214a-8a2d-43ee-bd6e-ea5a3592dfe6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:11.342261 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6f49214a-8a2d-43ee-bd6e-ea5a3592dfe6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:11.476886 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6f49214a-8a2d-43ee-bd6e-ea5a3592dfe6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:11.484911 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6f49214a-8a2d-43ee-bd6e-ea5a3592dfe6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1319 microversion: 2.1 time: 0.155887 May 28 03:40:11.485515 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 309/601] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:11 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1319 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:11.591492 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-92856abc-d0df-4417-9c20-1326f8d96a5b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:11.592518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-92856abc-d0df-4417-9c20-1326f8d96a5b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:11.596014 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d4e9e73e-9296-44a8-bc92-85f44aeb5e9c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:11.596623 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d4e9e73e-9296-44a8-bc92-85f44aeb5e9c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:11.605914 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-92856abc-d0df-4417-9c20-1326f8d96a5b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:11.605914 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-92856abc-d0df-4417-9c20-1326f8d96a5b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:11.605914 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-92856abc-d0df-4417-9c20-1326f8d96a5b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:11.605914 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-92856abc-d0df-4417-9c20-1326f8d96a5b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:11.606391 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4e9e73e-9296-44a8-bc92-85f44aeb5e9c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:11.606391 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4e9e73e-9296-44a8-bc92-85f44aeb5e9c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:11.606391 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4e9e73e-9296-44a8-bc92-85f44aeb5e9c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:11.606391 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4e9e73e-9296-44a8-bc92-85f44aeb5e9c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:11.632081 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-92856abc-d0df-4417-9c20-1326f8d96a5b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044969 May 28 03:40:11.636488 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 293/602] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:11 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:40:11.735818 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d4e9e73e-9296-44a8-bc92-85f44aeb5e9c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:11.744163 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d4e9e73e-9296-44a8-bc92-85f44aeb5e9c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.151490 May 28 03:40:11.744783 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 310/603] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:11 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:12.234415 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8cd81aa0-dddc-483e-8054-bab473ea1074 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:12.234415 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8cd81aa0-dddc-483e-8054-bab473ea1074 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:12.248707 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8cd81aa0-dddc-483e-8054-bab473ea1074 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:12.248909 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8cd81aa0-dddc-483e-8054-bab473ea1074 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:12.249652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8cd81aa0-dddc-483e-8054-bab473ea1074 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:12.250086 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8cd81aa0-dddc-483e-8054-bab473ea1074 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:12.502436 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-80ed472f-e324-4607-b4cc-d86f1db84bb1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:12.503688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-80ed472f-e324-4607-b4cc-d86f1db84bb1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:12.510858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-80ed472f-e324-4607-b4cc-d86f1db84bb1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:12.511302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-80ed472f-e324-4607-b4cc-d86f1db84bb1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:12.513416 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-80ed472f-e324-4607-b4cc-d86f1db84bb1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:12.513416 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-80ed472f-e324-4607-b4cc-d86f1db84bb1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:12.549214 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8cd81aa0-dddc-483e-8054-bab473ea1074 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:12.559269 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8cd81aa0-dddc-483e-8054-bab473ea1074 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.329045 May 28 03:40:12.560079 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 294/604] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:12 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:12.644130 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1086e82f-fd8e-4903-b58c-0bf9e98a0191 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:12.645087 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1086e82f-fd8e-4903-b58c-0bf9e98a0191 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:12.649360 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-80ed472f-e324-4607-b4cc-d86f1db84bb1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:12.658984 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-80ed472f-e324-4607-b4cc-d86f1db84bb1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1319 microversion: 2.1 time: 0.160506 May 28 03:40:12.659543 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 311/605] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:12 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1319 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:12.666728 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1086e82f-fd8e-4903-b58c-0bf9e98a0191 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:12.666728 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1086e82f-fd8e-4903-b58c-0bf9e98a0191 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:12.666728 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1086e82f-fd8e-4903-b58c-0bf9e98a0191 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:12.666728 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1086e82f-fd8e-4903-b58c-0bf9e98a0191 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:12.694627 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1086e82f-fd8e-4903-b58c-0bf9e98a0191 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.054550 May 28 03:40:12.695303 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 295/606] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:40:12 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547/os-volume_attachments => generated 25 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:12.763333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2b5a8c23-3054-4d04-812b-c0a893124586 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:12.763846 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2b5a8c23-3054-4d04-812b-c0a893124586 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:12.776145 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2b5a8c23-3054-4d04-812b-c0a893124586 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:12.777778 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2b5a8c23-3054-4d04-812b-c0a893124586 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:12.778893 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-41e3e715-6406-48fd-81a5-a7e3fc33244a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:12.779642 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2b5a8c23-3054-4d04-812b-c0a893124586 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:12.780966 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2b5a8c23-3054-4d04-812b-c0a893124586 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:12.782555 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-41e3e715-6406-48fd-81a5-a7e3fc33244a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:12.789850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-41e3e715-6406-48fd-81a5-a7e3fc33244a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:12.801824 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-41e3e715-6406-48fd-81a5-a7e3fc33244a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:12.804973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-41e3e715-6406-48fd-81a5-a7e3fc33244a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:12.806814 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-41e3e715-6406-48fd-81a5-a7e3fc33244a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:12.820627 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2b5a8c23-3054-4d04-812b-c0a893124586 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Going to try to terminate instance {{(pid=124604) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:40:12.858808 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:12.973692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-41e3e715-6406-48fd-81a5-a7e3fc33244a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:12.982803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-41e3e715-6406-48fd-81a5-a7e3fc33244a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.215297 May 28 03:40:12.983367 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 296/607] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:12 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:13.008335 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-3b1dc31b-1254-4492-8214-84388c03cac6 req-a152d957-7db8-45c3-b497-c17f633c8849 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1eaf0b19-b250-4970-adb1-1cdad09fff17", "tag": "8fe82109-26bb-498f-a1a1-825762536457"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:13.024205 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-3b1dc31b-1254-4492-8214-84388c03cac6 req-a152d957-7db8-45c3-b497-c17f633c8849 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:13.024755 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-3b1dc31b-1254-4492-8214-84388c03cac6 req-a152d957-7db8-45c3-b497-c17f633c8849 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:13.036638 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-3b1dc31b-1254-4492-8214-84388c03cac6 req-a152d957-7db8-45c3-b497-c17f633c8849 service nova] Creating event network-changed:8fe82109-26bb-498f-a1a1-825762536457 for instance 1eaf0b19-b250-4970-adb1-1cdad09fff17 on hv-cinder-84336 May 28 03:40:13.042101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-3b1dc31b-1254-4492-8214-84388c03cac6 req-a152d957-7db8-45c3-b497-c17f633c8849 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049139 May 28 03:40:13.042684 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 297/608] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:13.104097 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2b5a8c23-3054-4d04-812b-c0a893124586 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "DELETE /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 204 len: 0 microversion: 2.20 time: 0.345052 May 28 03:40:13.104097 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 312/609] 10.222.0.41 () {60 vars in 1279 bytes} [Sat May 28 03:40:12 2022] DELETE /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 28 03:40:13.112444 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3793e3d6-eef8-4902-b710-b63467b08e54 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:13.112875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3793e3d6-eef8-4902-b710-b63467b08e54 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:13.120009 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3793e3d6-eef8-4902-b710-b63467b08e54 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:13.120452 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3793e3d6-eef8-4902-b710-b63467b08e54 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:13.121256 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3793e3d6-eef8-4902-b710-b63467b08e54 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:13.121690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3793e3d6-eef8-4902-b710-b63467b08e54 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:13.254695 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3793e3d6-eef8-4902-b710-b63467b08e54 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:13.262796 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3793e3d6-eef8-4902-b710-b63467b08e54 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:13.264382 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3793e3d6-eef8-4902-b710-b63467b08e54 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:13.265577 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3793e3d6-eef8-4902-b710-b63467b08e54 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.157725 May 28 03:40:13.266127 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 298/610] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:13 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:13.589022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-64dd19d1-c248-45da-a949-f47c1e55ea4b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:13.589022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-64dd19d1-c248-45da-a949-f47c1e55ea4b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:13.589022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-64dd19d1-c248-45da-a949-f47c1e55ea4b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:13.589022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-64dd19d1-c248-45da-a949-f47c1e55ea4b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:13.590214 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-64dd19d1-c248-45da-a949-f47c1e55ea4b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:13.590214 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-64dd19d1-c248-45da-a949-f47c1e55ea4b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:13.675800 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2abc66d5-8ac7-449e-8472-32db93629d4f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:13.676553 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2abc66d5-8ac7-449e-8472-32db93629d4f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:13.684005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2abc66d5-8ac7-449e-8472-32db93629d4f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:13.684450 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2abc66d5-8ac7-449e-8472-32db93629d4f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:13.685234 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2abc66d5-8ac7-449e-8472-32db93629d4f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:13.685663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2abc66d5-8ac7-449e-8472-32db93629d4f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:13.722689 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-64dd19d1-c248-45da-a949-f47c1e55ea4b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:13.732590 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-64dd19d1-c248-45da-a949-f47c1e55ea4b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.159152 May 28 03:40:13.733424 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 313/611] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:13 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:13.817613 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2abc66d5-8ac7-449e-8472-32db93629d4f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:13.825038 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2abc66d5-8ac7-449e-8472-32db93629d4f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1319 microversion: 2.1 time: 0.153089 May 28 03:40:13.825654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 299/612] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:13 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1319 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:14.000227 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-fb68d625-1836-49a6-b748-9d697dfe26b3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:14.000838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-fb68d625-1836-49a6-b748-9d697dfe26b3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:14.006929 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fb68d625-1836-49a6-b748-9d697dfe26b3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:14.007377 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fb68d625-1836-49a6-b748-9d697dfe26b3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:14.008137 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fb68d625-1836-49a6-b748-9d697dfe26b3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:14.008573 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fb68d625-1836-49a6-b748-9d697dfe26b3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:14.145635 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-fb68d625-1836-49a6-b748-9d697dfe26b3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:14.153130 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-fb68d625-1836-49a6-b748-9d697dfe26b3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.156741 May 28 03:40:14.153695 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 314/613] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:13 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:14.283458 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f3a3c8b9-e891-47a0-a68b-f596db7ebb16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:14.284072 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f3a3c8b9-e891-47a0-a68b-f596db7ebb16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:14.290301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3a3c8b9-e891-47a0-a68b-f596db7ebb16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:14.290779 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3a3c8b9-e891-47a0-a68b-f596db7ebb16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:14.292093 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3a3c8b9-e891-47a0-a68b-f596db7ebb16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:14.292599 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f3a3c8b9-e891-47a0-a68b-f596db7ebb16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:14.418788 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f3a3c8b9-e891-47a0-a68b-f596db7ebb16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:14.435334 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f3a3c8b9-e891-47a0-a68b-f596db7ebb16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:14.436811 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f3a3c8b9-e891-47a0-a68b-f596db7ebb16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:14.437990 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f3a3c8b9-e891-47a0-a68b-f596db7ebb16 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.158429 May 28 03:40:14.438546 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 300/614] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:14 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:14.750261 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d5bac995-a8ba-401e-bde4-9b06a2fe623d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:14.750906 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d5bac995-a8ba-401e-bde4-9b06a2fe623d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:14.757819 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5bac995-a8ba-401e-bde4-9b06a2fe623d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:14.758294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5bac995-a8ba-401e-bde4-9b06a2fe623d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:14.759066 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5bac995-a8ba-401e-bde4-9b06a2fe623d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:14.760035 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5bac995-a8ba-401e-bde4-9b06a2fe623d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:14.846162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8c9ebad2-5c7b-4c19-91f6-414dc38e6158 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:14.846778 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8c9ebad2-5c7b-4c19-91f6-414dc38e6158 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:14.854086 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c9ebad2-5c7b-4c19-91f6-414dc38e6158 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:14.856614 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c9ebad2-5c7b-4c19-91f6-414dc38e6158 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:14.856614 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c9ebad2-5c7b-4c19-91f6-414dc38e6158 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:14.856614 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c9ebad2-5c7b-4c19-91f6-414dc38e6158 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:14.903968 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d5bac995-a8ba-401e-bde4-9b06a2fe623d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:14.913500 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d5bac995-a8ba-401e-bde4-9b06a2fe623d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.166972 May 28 03:40:14.914203 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 315/615] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:14 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:15.170532 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2591fd65-7911-450f-a8f0-2884e16886b6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:15.171191 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2591fd65-7911-450f-a8f0-2884e16886b6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:15.177913 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2591fd65-7911-450f-a8f0-2884e16886b6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:15.178411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2591fd65-7911-450f-a8f0-2884e16886b6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:15.179283 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2591fd65-7911-450f-a8f0-2884e16886b6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:15.179772 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2591fd65-7911-450f-a8f0-2884e16886b6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:15.330845 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-2591fd65-7911-450f-a8f0-2884e16886b6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:15.344074 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2591fd65-7911-450f-a8f0-2884e16886b6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.177653 May 28 03:40:15.345066 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 316/616] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:15 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:15.405186 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8c9ebad2-5c7b-4c19-91f6-414dc38e6158 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:15.407330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-71906234-4130-4c71-9128-61c841a32e6f req-e964b127-c41b-4047-a08e-d5a884b8346a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c2d3a613-c1f1-49d5-a707-784bdc430953", "name": "network-vif-plugged", "status": "completed", "tag": "f1321a9e-949d-4d52-abcc-e48ee7436a3a"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:15.416362 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-71906234-4130-4c71-9128-61c841a32e6f req-e964b127-c41b-4047-a08e-d5a884b8346a service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:15.417084 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-71906234-4130-4c71-9128-61c841a32e6f req-e964b127-c41b-4047-a08e-d5a884b8346a service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:15.417225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8c9ebad2-5c7b-4c19-91f6-414dc38e6158 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1319 microversion: 2.1 time: 0.573538 May 28 03:40:15.422967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 301/617] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:14 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1319 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:15.432297 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-71906234-4130-4c71-9128-61c841a32e6f req-e964b127-c41b-4047-a08e-d5a884b8346a service nova] Creating event network-vif-plugged:f1321a9e-949d-4d52-abcc-e48ee7436a3a for instance c2d3a613-c1f1-49d5-a707-784bdc430953 on hv-cinder-84336 May 28 03:40:15.437352 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-71906234-4130-4c71-9128-61c841a32e6f req-e964b127-c41b-4047-a08e-d5a884b8346a service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033606 May 28 03:40:15.438447 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 317/618] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:40:15.460790 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e4881060-2f89-44c4-a720-c4dbdacda3f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:15.461959 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e4881060-2f89-44c4-a720-c4dbdacda3f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:15.468509 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4881060-2f89-44c4-a720-c4dbdacda3f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:15.468956 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4881060-2f89-44c4-a720-c4dbdacda3f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:15.469798 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4881060-2f89-44c4-a720-c4dbdacda3f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:15.470235 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4881060-2f89-44c4-a720-c4dbdacda3f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:15.808370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e4881060-2f89-44c4-a720-c4dbdacda3f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:15.817376 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e4881060-2f89-44c4-a720-c4dbdacda3f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:15.818895 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e4881060-2f89-44c4-a720-c4dbdacda3f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:15.820100 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e4881060-2f89-44c4-a720-c4dbdacda3f3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1725 microversion: 2.20 time: 0.364290 May 28 03:40:15.831448 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 302/619] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:15 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1725 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:15.932381 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a0a34934-3b39-4ea4-bd9a-0de9ab502b26 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:15.933769 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a0a34934-3b39-4ea4-bd9a-0de9ab502b26 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:15.940029 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a0a34934-3b39-4ea4-bd9a-0de9ab502b26 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:15.940471 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a0a34934-3b39-4ea4-bd9a-0de9ab502b26 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:15.941461 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a0a34934-3b39-4ea4-bd9a-0de9ab502b26 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:15.941900 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a0a34934-3b39-4ea4-bd9a-0de9ab502b26 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:16.095616 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a0a34934-3b39-4ea4-bd9a-0de9ab502b26 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:16.104260 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a0a34934-3b39-4ea4-bd9a-0de9ab502b26 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.176410 May 28 03:40:16.104836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 318/620] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:15 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:16.361392 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8ab53d6e-4a22-4314-9939-7a3498a79c3b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:16.362005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8ab53d6e-4a22-4314-9939-7a3498a79c3b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:16.368718 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8ab53d6e-4a22-4314-9939-7a3498a79c3b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:16.369760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8ab53d6e-4a22-4314-9939-7a3498a79c3b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:16.370534 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8ab53d6e-4a22-4314-9939-7a3498a79c3b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:16.370972 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8ab53d6e-4a22-4314-9939-7a3498a79c3b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:16.389003 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=100 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:16.434322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-85741a56-da65-4bae-b76a-aab9fb2eeb23 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:16.435270 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-85741a56-da65-4bae-b76a-aab9fb2eeb23 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:16.441848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-85741a56-da65-4bae-b76a-aab9fb2eeb23 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:16.442224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-85741a56-da65-4bae-b76a-aab9fb2eeb23 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:16.443007 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-85741a56-da65-4bae-b76a-aab9fb2eeb23 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:16.443449 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-85741a56-da65-4bae-b76a-aab9fb2eeb23 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:16.576984 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-85741a56-da65-4bae-b76a-aab9fb2eeb23 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:16.585952 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-85741a56-da65-4bae-b76a-aab9fb2eeb23 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1319 microversion: 2.1 time: 0.156008 May 28 03:40:16.586749 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 319/621] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:16 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:16.614584 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8ab53d6e-4a22-4314-9939-7a3498a79c3b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:16.657205 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8ab53d6e-4a22-4314-9939-7a3498a79c3b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.297268 May 28 03:40:16.665620 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 303/622] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:16 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:16.684846 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-b818d4ee-0092-4c75-8b10-12ec89b6b3af req-e579827e-5d99-4d8a-86b0-421f5c5af4b4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35a9c645-590f-49e1-ab03-af4167f10547", "name": "network-vif-deleted", "tag": "97df7472-c1e5-4bbd-80b5-6efb1965aeb9"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:16.693374 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-b818d4ee-0092-4c75-8b10-12ec89b6b3af req-e579827e-5d99-4d8a-86b0-421f5c5af4b4 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:16.693740 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-b818d4ee-0092-4c75-8b10-12ec89b6b3af req-e579827e-5d99-4d8a-86b0-421f5c5af4b4 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:16.707052 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-b818d4ee-0092-4c75-8b10-12ec89b6b3af req-e579827e-5d99-4d8a-86b0-421f5c5af4b4 service nova] Creating event network-vif-deleted:97df7472-c1e5-4bbd-80b5-6efb1965aeb9 for instance 35a9c645-590f-49e1-ab03-af4167f10547 on hv-cinder-84336 May 28 03:40:16.711678 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-b818d4ee-0092-4c75-8b10-12ec89b6b3af req-e579827e-5d99-4d8a-86b0-421f5c5af4b4 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031002 May 28 03:40:16.712292 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 320/623] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:16.839091 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3efa3c27-6ec3-4dd9-afd7-5ec507ec3190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:16.839631 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3efa3c27-6ec3-4dd9-afd7-5ec507ec3190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:16.845802 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3efa3c27-6ec3-4dd9-afd7-5ec507ec3190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:16.846251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3efa3c27-6ec3-4dd9-afd7-5ec507ec3190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:16.847030 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3efa3c27-6ec3-4dd9-afd7-5ec507ec3190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:16.847515 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3efa3c27-6ec3-4dd9-afd7-5ec507ec3190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:16.923518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3efa3c27-6ec3-4dd9-afd7-5ec507ec3190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:16.929128 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3efa3c27-6ec3-4dd9-afd7-5ec507ec3190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:16.930826 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3efa3c27-6ec3-4dd9-afd7-5ec507ec3190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:16.932229 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3efa3c27-6ec3-4dd9-afd7-5ec507ec3190 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1657 microversion: 2.20 time: 0.097703 May 28 03:40:16.933554 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 304/624] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:16 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1657 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:40:17.122388 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ff2a4798-94b7-4b05-b8b5-db3ae74c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:17.123217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ff2a4798-94b7-4b05-b8b5-db3ae74c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:17.129592 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ff2a4798-94b7-4b05-b8b5-db3ae74c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:17.130104 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ff2a4798-94b7-4b05-b8b5-db3ae74c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:17.130881 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ff2a4798-94b7-4b05-b8b5-db3ae74c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:17.131318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ff2a4798-94b7-4b05-b8b5-db3ae74c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:17.286305 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ff2a4798-94b7-4b05-b8b5-db3ae74c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:17.295102 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ff2a4798-94b7-4b05-b8b5-db3ae74c2829 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.177336 May 28 03:40:17.295682 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 321/625] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:17 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:17.604000 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-948df0a2-f6a1-4513-b5a6-b1516fd4604b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:17.605277 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-948df0a2-f6a1-4513-b5a6-b1516fd4604b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:17.611687 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-948df0a2-f6a1-4513-b5a6-b1516fd4604b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:17.612129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-948df0a2-f6a1-4513-b5a6-b1516fd4604b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:17.612886 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-948df0a2-f6a1-4513-b5a6-b1516fd4604b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:17.613341 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-948df0a2-f6a1-4513-b5a6-b1516fd4604b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:17.673635 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bafe6995-1208-4268-9a2a-83e3c8d3cdfe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:17.674356 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bafe6995-1208-4268-9a2a-83e3c8d3cdfe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:17.685601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bafe6995-1208-4268-9a2a-83e3c8d3cdfe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:17.688237 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bafe6995-1208-4268-9a2a-83e3c8d3cdfe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:17.689179 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bafe6995-1208-4268-9a2a-83e3c8d3cdfe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:17.690737 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bafe6995-1208-4268-9a2a-83e3c8d3cdfe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:17.760518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-948df0a2-f6a1-4513-b5a6-b1516fd4604b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:17.768992 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-948df0a2-f6a1-4513-b5a6-b1516fd4604b tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1319 microversion: 2.1 time: 0.169450 May 28 03:40:17.769610 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 305/626] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:17 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1319 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:17.843039 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bafe6995-1208-4268-9a2a-83e3c8d3cdfe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:17.850471 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bafe6995-1208-4268-9a2a-83e3c8d3cdfe tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1367 microversion: 2.1 time: 0.180773 May 28 03:40:17.854397 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 322/627] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:17 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1367 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:17.950921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:17.951555 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:17.957573 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:17.958022 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:17.958791 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:17.959227 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:17.989660 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:17.990533 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:17.990533 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lazy-loading 'fault' on Instance uuid 35a9c645-590f-49e1-ab03-af4167f10547 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:40:18.032223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:18.039989 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:18.042326 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:18.047941 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-698d2418-d20b-4177-b752-c12fcf8628f1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 200 len: 1372 microversion: 2.20 time: 0.096504 May 28 03:40:18.047941 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 306/628] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:17 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:18.312869 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-02711244-fe00-4eae-9389-699e7315f922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:18.313512 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-02711244-fe00-4eae-9389-699e7315f922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:18.319969 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02711244-fe00-4eae-9389-699e7315f922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:18.320420 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02711244-fe00-4eae-9389-699e7315f922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:18.321669 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02711244-fe00-4eae-9389-699e7315f922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:18.321669 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02711244-fe00-4eae-9389-699e7315f922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:18.458191 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-02711244-fe00-4eae-9389-699e7315f922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:18.466760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-02711244-fe00-4eae-9389-699e7315f922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.157777 May 28 03:40:18.467328 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 323/629] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:18 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:18.786488 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7a77e516-850b-4ac5-849e-ac425eb2cec1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:18.787368 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7a77e516-850b-4ac5-849e-ac425eb2cec1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:18.793991 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7a77e516-850b-4ac5-849e-ac425eb2cec1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:18.794442 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7a77e516-850b-4ac5-849e-ac425eb2cec1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:18.795207 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7a77e516-850b-4ac5-849e-ac425eb2cec1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:18.795640 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7a77e516-850b-4ac5-849e-ac425eb2cec1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:18.868143 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d4f5b423-95d6-40e7-93c6-48d326929443 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:18.868672 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d4f5b423-95d6-40e7-93c6-48d326929443 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:18.875323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4f5b423-95d6-40e7-93c6-48d326929443 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:18.875762 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4f5b423-95d6-40e7-93c6-48d326929443 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:18.876531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4f5b423-95d6-40e7-93c6-48d326929443 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:18.876962 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4f5b423-95d6-40e7-93c6-48d326929443 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:18.933178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7a77e516-850b-4ac5-849e-ac425eb2cec1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:18.940902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7a77e516-850b-4ac5-849e-ac425eb2cec1 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1319 microversion: 2.1 time: 0.158186 May 28 03:40:18.941933 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 307/630] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:18 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1319 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:19.061366 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-fd62f8c6-622f-45b4-b562-77aedc5f9582 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:19.062011 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-fd62f8c6-622f-45b4-b562-77aedc5f9582 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:19.068271 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fd62f8c6-622f-45b4-b562-77aedc5f9582 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:19.068752 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fd62f8c6-622f-45b4-b562-77aedc5f9582 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:19.069595 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fd62f8c6-622f-45b4-b562-77aedc5f9582 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:19.070047 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fd62f8c6-622f-45b4-b562-77aedc5f9582 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:19.084047 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-fd62f8c6-622f-45b4-b562-77aedc5f9582 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] HTTP exception thrown: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-fd62f8c6-622f-45b4-b562-77aedc5f9582 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Request method failure captured: May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 HTTP/1.1 May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZiXoa35A6jROnZ8zMQef32Be4oOIf-urjr-I_rz1MgM2_b3S2PmayB1X84xgl-OkycZsqj_AQ7q_z4FJL3oFFaOgR07hPjMPlbq-6NjLaC0eWnozGEBu56f-_apeayHLhoZJdjUAE9qXMfIOywazcSomdQ-kwUGKS-fFaJNB3natDQ May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Openstack-Nova-Api-Version: 2.20 May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:40:19.085732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: e0eb0ab5585747d098bbb0b18770f4c5 May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:40:19.087386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '35a9c645-590f-49e1-ab03-af4167f10547'} May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:40:19.092301 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:40:19.093829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:19.095169 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-fd62f8c6-622f-45b4-b562-77aedc5f9582 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Returning 404 to user: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:40:19.095169 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-fd62f8c6-622f-45b4-b562-77aedc5f9582 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 404 len: 111 microversion: 2.20 time: 0.029783 May 28 03:40:19.095169 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 308/631] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:19 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 28 03:40:19.096482 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4388e0c1-e834-46c4-983a-6b86286ad3a9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '.version_select at 0x7feb1fc1eb80>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:19.216497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d4f5b423-95d6-40e7-93c6-48d326929443 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:19.224058 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d4f5b423-95d6-40e7-93c6-48d326929443 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1556 microversion: 2.1 time: 0.360213 May 28 03:40:19.235768 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 324/632] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:18 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1556 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:19.236134 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4388e0c1-e834-46c4-983a-6b86286ad3a9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2087163104" status: 204 len: 0 microversion: 2.20 time: 0.135643 May 28 03:40:19.236134 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 309/633] 10.222.0.41 () {60 vars in 1261 bytes} [Sat May 28 03:40:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2087163104 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) May 28 03:40:19.242194 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-36d9e9d9-ca08-43f5-a85f-d9e3dd76c9b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:19.245785 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-36d9e9d9-ca08-43f5-a85f-d9e3dd76c9b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:19.246992 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:19.250946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-36d9e9d9-ca08-43f5-a85f-d9e3dd76c9b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:19.251394 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-36d9e9d9-ca08-43f5-a85f-d9e3dd76c9b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:19.252176 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-36d9e9d9-ca08-43f5-a85f-d9e3dd76c9b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:19.252689 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-36d9e9d9-ca08-43f5-a85f-d9e3dd76c9b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:19.351754 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-36d9e9d9-ca08-43f5-a85f-d9e3dd76c9b4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.113934 May 28 03:40:19.354399 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 325/634] 10.222.0.41 () {58 vars in 1274 bytes} [Sat May 28 03:40:19 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-interface => generated 284 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:19.484574 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8f7892be-0a27-4d97-a04a-5687a0505dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:19.491592 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8f7892be-0a27-4d97-a04a-5687a0505dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:19.500067 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8f7892be-0a27-4d97-a04a-5687a0505dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:19.500700 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8f7892be-0a27-4d97-a04a-5687a0505dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:19.501578 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8f7892be-0a27-4d97-a04a-5687a0505dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:19.502482 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8f7892be-0a27-4d97-a04a-5687a0505dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:19.786321 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8f7892be-0a27-4d97-a04a-5687a0505dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:19.794663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8f7892be-0a27-4d97-a04a-5687a0505dc7 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.314236 May 28 03:40:19.795332 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 310/635] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:19 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:19.958936 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e2a4d90a-ad77-4331-b831-80bb13a8f96c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:19.962560 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e2a4d90a-ad77-4331-b831-80bb13a8f96c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:19.977062 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2a4d90a-ad77-4331-b831-80bb13a8f96c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:19.979777 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2a4d90a-ad77-4331-b831-80bb13a8f96c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:19.979777 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2a4d90a-ad77-4331-b831-80bb13a8f96c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:19.979777 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2a4d90a-ad77-4331-b831-80bb13a8f96c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:20.191313 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e2a4d90a-ad77-4331-b831-80bb13a8f96c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:20.199940 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e2a4d90a-ad77-4331-b831-80bb13a8f96c tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1319 microversion: 2.1 time: 0.244991 May 28 03:40:20.209140 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 326/636] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:19 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1319 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:20.247497 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b3ca0351-8e8a-400c-8400-0d4a062b281d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Action: 'create', calling method: .version_select at 0x7feb1fc1e5e0>, body: {"keypair": {"name": "tempest-keypair-183206554", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNTJFqKuF2YICk2NbiXSi3IICBDRoesN4WTYSsTxGumd5B4UIFyL2XBP+1PSE+XVf571u4NDnOvHzNLZPP2LXZaAcy5Trc29iEF4PF4vtlyZFJTeq1IDK96xcZrJQaNWCQ=="}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:20.261426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-b3ca0351-8e8a-400c-8400-0d4a062b281d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Getting quotas for project 3611c72fb0b8472da98ba9b0c0d86843. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:40:20.265185 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-b3ca0351-8e8a-400c-8400-0d4a062b281d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Getting quotas for user e0eb0ab5585747d098bbb0b18770f4c5 and project 3611c72fb0b8472da98ba9b0c0d86843. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:40:20.267394 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=99,nova_api:DELETE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:20.354101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-b3ca0351-8e8a-400c-8400-0d4a062b281d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.110959 May 28 03:40:20.354704 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 311/637] 10.222.0.41 () {62 vars in 1199 bytes} [Sat May 28 03:40:20 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 113 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) May 28 03:40:20.627518 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-e48e2a37-20fd-4048-9612-a41ff5073202 req-13ae9e06-3259-4528-b403-6b098b0fa03c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c2d3a613-c1f1-49d5-a707-784bdc430953", "tag": "f1321a9e-949d-4d52-abcc-e48ee7436a3a"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:20.637039 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-e48e2a37-20fd-4048-9612-a41ff5073202 req-13ae9e06-3259-4528-b403-6b098b0fa03c service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:20.637754 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-e48e2a37-20fd-4048-9612-a41ff5073202 req-13ae9e06-3259-4528-b403-6b098b0fa03c service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:20.649206 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-e48e2a37-20fd-4048-9612-a41ff5073202 req-13ae9e06-3259-4528-b403-6b098b0fa03c service nova] Creating event network-changed:f1321a9e-949d-4d52-abcc-e48ee7436a3a for instance c2d3a613-c1f1-49d5-a707-784bdc430953 on hv-cinder-84336 May 28 03:40:20.655089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-e48e2a37-20fd-4048-9612-a41ff5073202 req-13ae9e06-3259-4528-b403-6b098b0fa03c service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031305 May 28 03:40:20.655735 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 327/638] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:20.813709 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-76df38d1-bded-4719-8774-16808504135b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:20.814326 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-76df38d1-bded-4719-8774-16808504135b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:20.820864 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-76df38d1-bded-4719-8774-16808504135b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:20.821587 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-76df38d1-bded-4719-8774-16808504135b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:20.823226 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-76df38d1-bded-4719-8774-16808504135b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:20.823652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-76df38d1-bded-4719-8774-16808504135b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:21.074272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-76df38d1-bded-4719-8774-16808504135b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:21.088343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-76df38d1-bded-4719-8774-16808504135b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.279306 May 28 03:40:21.089693 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 312/639] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:20 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:21.223168 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-49ef76f4-96dd-42cc-b3c4-a339566f9065 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:21.223884 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-49ef76f4-96dd-42cc-b3c4-a339566f9065 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:21.242178 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49ef76f4-96dd-42cc-b3c4-a339566f9065 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:21.242629 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49ef76f4-96dd-42cc-b3c4-a339566f9065 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:21.254698 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49ef76f4-96dd-42cc-b3c4-a339566f9065 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:21.254698 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49ef76f4-96dd-42cc-b3c4-a339566f9065 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:21.430317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-49ef76f4-96dd-42cc-b3c4-a339566f9065 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:21.438424 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-49ef76f4-96dd-42cc-b3c4-a339566f9065 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1319 microversion: 2.1 time: 0.226129 May 28 03:40:21.450427 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 328/640] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:21 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1319 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:21.526298 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-bd52d4f2-bdec-4565-a72d-b42f4177baa0 req-26b02204-31bf-4f98-8bf9-db29cc94415d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1eaf0b19-b250-4970-adb1-1cdad09fff17", "name": "network-vif-plugged", "status": "completed", "tag": "8fe82109-26bb-498f-a1a1-825762536457"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:21.541257 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-bd52d4f2-bdec-4565-a72d-b42f4177baa0 req-26b02204-31bf-4f98-8bf9-db29cc94415d service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:21.541745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-bd52d4f2-bdec-4565-a72d-b42f4177baa0 req-26b02204-31bf-4f98-8bf9-db29cc94415d service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:21.556387 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-bd52d4f2-bdec-4565-a72d-b42f4177baa0 req-26b02204-31bf-4f98-8bf9-db29cc94415d service nova] Creating event network-vif-plugged:8fe82109-26bb-498f-a1a1-825762536457 for instance 1eaf0b19-b250-4970-adb1-1cdad09fff17 on hv-cinder-84336 May 28 03:40:21.562679 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-bd52d4f2-bdec-4565-a72d-b42f4177baa0 req-26b02204-31bf-4f98-8bf9-db29cc94415d service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041415 May 28 03:40:21.563393 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 313/641] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:22.106540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-59ce436f-0ca6-4e79-980e-ec0b2f0f0ce6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:22.107146 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-59ce436f-0ca6-4e79-980e-ec0b2f0f0ce6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:22.113558 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-59ce436f-0ca6-4e79-980e-ec0b2f0f0ce6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:22.114002 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-59ce436f-0ca6-4e79-980e-ec0b2f0f0ce6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:22.114769 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-59ce436f-0ca6-4e79-980e-ec0b2f0f0ce6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:22.115190 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-59ce436f-0ca6-4e79-980e-ec0b2f0f0ce6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:22.254835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-59ce436f-0ca6-4e79-980e-ec0b2f0f0ce6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:22.263890 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-59ce436f-0ca6-4e79-980e-ec0b2f0f0ce6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.161663 May 28 03:40:22.266488 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 329/642] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:22 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:22.456896 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7d2b4d76-e06f-44d7-b804-e8926eb84555 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:22.457775 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7d2b4d76-e06f-44d7-b804-e8926eb84555 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:22.464175 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d2b4d76-e06f-44d7-b804-e8926eb84555 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:22.464683 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d2b4d76-e06f-44d7-b804-e8926eb84555 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:22.465563 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d2b4d76-e06f-44d7-b804-e8926eb84555 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:22.466089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d2b4d76-e06f-44d7-b804-e8926eb84555 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:22.699049 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7d2b4d76-e06f-44d7-b804-e8926eb84555 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:22.706435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7d2b4d76-e06f-44d7-b804-e8926eb84555 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1319 microversion: 2.1 time: 0.254313 May 28 03:40:22.707007 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 314/643] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:22 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1319 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:23.282016 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5bb754f2-3b8b-431b-b369-1b5dcee7780f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:23.282654 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5bb754f2-3b8b-431b-b369-1b5dcee7780f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:23.289399 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5bb754f2-3b8b-431b-b369-1b5dcee7780f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:23.289892 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5bb754f2-3b8b-431b-b369-1b5dcee7780f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:23.291367 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5bb754f2-3b8b-431b-b369-1b5dcee7780f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:23.291803 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5bb754f2-3b8b-431b-b369-1b5dcee7780f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:23.393843 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-220644160", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f47d9414-85e4-4030-814e-0a5133c3d4b0"}], "security_groups": [{"name": "tempest-securitygroup--1685871804"}], "key_name": "tempest-keypair-183206554", "user_data": "}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:23.439107 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5bb754f2-3b8b-431b-b369-1b5dcee7780f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:23.447306 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5bb754f2-3b8b-431b-b369-1b5dcee7780f tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.169222 May 28 03:40:23.447872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 330/644] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:23 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:23.495650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.network.neutron [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] validate_networks() for [('f47d9414-85e4-4030-814e-0a5133c3d4b0', None, None, None, None, None)] {{(pid=124605) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:40:23.722371 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8a55cb1b-b920-4b2d-a29b-166a4f49bd09 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:23.722982 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8a55cb1b-b920-4b2d-a29b-166a4f49bd09 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:23.729086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8a55cb1b-b920-4b2d-a29b-166a4f49bd09 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:23.729572 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8a55cb1b-b920-4b2d-a29b-166a4f49bd09 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:23.730339 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8a55cb1b-b920-4b2d-a29b-166a4f49bd09 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:23.730774 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8a55cb1b-b920-4b2d-a29b-166a4f49bd09 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:23.734965 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:23.867548 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8a55cb1b-b920-4b2d-a29b-166a4f49bd09 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:23.875097 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Flavor limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:40:23.875509 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8a55cb1b-b920-4b2d-a29b-166a4f49bd09 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1508 microversion: 2.1 time: 0.156959 May 28 03:40:23.875655 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Image limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:40:23.876093 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 331/645] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:23 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1508 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:23.876192 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Flavor pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:40:23.876560 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Image pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:40:23.876978 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:40:23.885900 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:23.886353 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:23.887148 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:23.887584 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:23.892023 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4e0a975c-8999-4cfd-abd0-3379b087dd81 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bd590ac9-b81b-4c8e-975d-ee237eda6c4e", "device": "/dev/sdb"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:23.893277 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4e0a975c-8999-4cfd-abd0-3379b087dd81 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:23.899347 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4e0a975c-8999-4cfd-abd0-3379b087dd81 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:23.899803 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4e0a975c-8999-4cfd-abd0-3379b087dd81 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:23.900581 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4e0a975c-8999-4cfd-abd0-3379b087dd81 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:23.901022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4e0a975c-8999-4cfd-abd0-3379b087dd81 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:23.904918 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Getting quotas for project 3611c72fb0b8472da98ba9b0c0d86843. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:40:23.909551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Getting quotas for user e0eb0ab5585747d098bbb0b18770f4c5 and project 3611c72fb0b8472da98ba9b0c0d86843. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:40:23.925648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Going to run 1 instances... {{(pid=124605) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:40:23.925648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:40:24.052018 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-62bdd118-cc55-45be-bd39-b0c805045cc1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.662584 May 28 03:40:24.052726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 315/646] 10.222.0.41 () {62 vars in 1187 bytes} [Sat May 28 03:40:23 2022] POST /compute/v2.1/servers => generated 402 bytes in 664 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) May 28 03:40:24.063472 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-877c448a-de81-4e9a-b9ea-80a8a2315f02 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:24.068615 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-877c448a-de81-4e9a-b9ea-80a8a2315f02 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:24.123086 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-877c448a-de81-4e9a-b9ea-80a8a2315f02 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:24.134361 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-877c448a-de81-4e9a-b9ea-80a8a2315f02 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:24.135877 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-877c448a-de81-4e9a-b9ea-80a8a2315f02 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:24.137155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-877c448a-de81-4e9a-b9ea-80a8a2315f02 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1280 microversion: 2.20 time: 0.077275 May 28 03:40:24.137718 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 316/647] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:24 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1280 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:40:24.465959 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-af1e3098-b165-4ffa-bd36-cf2d26bb1f2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:24.466605 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-af1e3098-b165-4ffa-bd36-cf2d26bb1f2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:24.473238 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-af1e3098-b165-4ffa-bd36-cf2d26bb1f2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:24.473696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-af1e3098-b165-4ffa-bd36-cf2d26bb1f2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:24.474458 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-af1e3098-b165-4ffa-bd36-cf2d26bb1f2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:24.475448 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-af1e3098-b165-4ffa-bd36-cf2d26bb1f2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:24.615739 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-af1e3098-b165-4ffa-bd36-cf2d26bb1f2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:24.623898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-af1e3098-b165-4ffa-bd36-cf2d26bb1f2d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.163372 May 28 03:40:24.624472 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 317/648] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:24 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:24.825930 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4e0a975c-8999-4cfd-abd0-3379b087dd81 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "POST /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.937728 May 28 03:40:24.826742 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 332/649] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:40:23 2022] POST /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17/os-volume_attachments => generated 194 bytes in 939 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:40:25.155509 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-fafa283f-417a-43ae-bf16-8559981ce700 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:25.156126 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-fafa283f-417a-43ae-bf16-8559981ce700 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:25.162088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fafa283f-417a-43ae-bf16-8559981ce700 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:25.162664 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fafa283f-417a-43ae-bf16-8559981ce700 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:25.163464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fafa283f-417a-43ae-bf16-8559981ce700 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:25.163924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fafa283f-417a-43ae-bf16-8559981ce700 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:25.226521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-fafa283f-417a-43ae-bf16-8559981ce700 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:25.235488 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-fafa283f-417a-43ae-bf16-8559981ce700 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:25.237089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-fafa283f-417a-43ae-bf16-8559981ce700 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:25.238207 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-fafa283f-417a-43ae-bf16-8559981ce700 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1284 microversion: 2.20 time: 0.086687 May 28 03:40:25.238753 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 318/650] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:25 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1284 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:25.642830 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4df7e471-ee7a-483b-b130-02a1eff5aff2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:25.643534 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4df7e471-ee7a-483b-b130-02a1eff5aff2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:25.649786 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4df7e471-ee7a-483b-b130-02a1eff5aff2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:25.650238 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4df7e471-ee7a-483b-b130-02a1eff5aff2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:25.651009 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4df7e471-ee7a-483b-b130-02a1eff5aff2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:25.651449 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4df7e471-ee7a-483b-b130-02a1eff5aff2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:25.780536 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4df7e471-ee7a-483b-b130-02a1eff5aff2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:25.788674 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4df7e471-ee7a-483b-b130-02a1eff5aff2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.150840 May 28 03:40:25.789275 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 333/651] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:25 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:26.259198 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2d345436-267c-4788-b6ac-3b1ef910ffc5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:26.259878 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2d345436-267c-4788-b6ac-3b1ef910ffc5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:26.267143 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2d345436-267c-4788-b6ac-3b1ef910ffc5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:26.267654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2d345436-267c-4788-b6ac-3b1ef910ffc5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:26.268438 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2d345436-267c-4788-b6ac-3b1ef910ffc5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:26.268877 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2d345436-267c-4788-b6ac-3b1ef910ffc5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:26.372176 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2d345436-267c-4788-b6ac-3b1ef910ffc5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:26.381366 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2d345436-267c-4788-b6ac-3b1ef910ffc5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:26.383119 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2d345436-267c-4788-b6ac-3b1ef910ffc5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:26.384242 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2d345436-267c-4788-b6ac-3b1ef910ffc5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1340 microversion: 2.20 time: 0.131408 May 28 03:40:26.384802 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 319/652] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:26 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1340 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:26.806616 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9c86f51c-6589-488a-a288-b0d397546759 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:26.807288 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9c86f51c-6589-488a-a288-b0d397546759 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:26.813982 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9c86f51c-6589-488a-a288-b0d397546759 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:26.814409 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9c86f51c-6589-488a-a288-b0d397546759 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:26.815168 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9c86f51c-6589-488a-a288-b0d397546759 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:26.815601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9c86f51c-6589-488a-a288-b0d397546759 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:26.952012 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9c86f51c-6589-488a-a288-b0d397546759 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:26.960881 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9c86f51c-6589-488a-a288-b0d397546759 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.158020 May 28 03:40:26.961905 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 334/653] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:26 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:27.402964 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cb6d2679-c1a7-4ec4-b18b-e7e62d852991 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:27.403590 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-cb6d2679-c1a7-4ec4-b18b-e7e62d852991 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:27.410241 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cb6d2679-c1a7-4ec4-b18b-e7e62d852991 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:27.410690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cb6d2679-c1a7-4ec4-b18b-e7e62d852991 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:27.411451 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cb6d2679-c1a7-4ec4-b18b-e7e62d852991 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:27.411883 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cb6d2679-c1a7-4ec4-b18b-e7e62d852991 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:27.581526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-cb6d2679-c1a7-4ec4-b18b-e7e62d852991 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:27.590190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-cb6d2679-c1a7-4ec4-b18b-e7e62d852991 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:27.591688 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-cb6d2679-c1a7-4ec4-b18b-e7e62d852991 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:27.592793 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-cb6d2679-c1a7-4ec4-b18b-e7e62d852991 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.193887 May 28 03:40:27.593374 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 320/654] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:27 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:27.978908 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8070749d-bd36-450a-b7d5-2baf522ec5e4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:27.979740 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8070749d-bd36-450a-b7d5-2baf522ec5e4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:27.987452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8070749d-bd36-450a-b7d5-2baf522ec5e4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:27.987974 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8070749d-bd36-450a-b7d5-2baf522ec5e4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:27.988883 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8070749d-bd36-450a-b7d5-2baf522ec5e4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:27.989598 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8070749d-bd36-450a-b7d5-2baf522ec5e4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:28.000727 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-b5f59ab7-8d6d-4e70-97c6-4a4b3a79b647 req-49083b6a-dda9-400e-88a4-4f64aff54cd8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6be2aeaa-8d90-4e51-87d2-d02a8566ac1a", "tag": "73f87b73-9d7c-43a7-9bd2-dc7fb5450112"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:28.015765 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-b5f59ab7-8d6d-4e70-97c6-4a4b3a79b647 req-49083b6a-dda9-400e-88a4-4f64aff54cd8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:28.016241 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-b5f59ab7-8d6d-4e70-97c6-4a4b3a79b647 req-49083b6a-dda9-400e-88a4-4f64aff54cd8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:28.028970 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-b5f59ab7-8d6d-4e70-97c6-4a4b3a79b647 req-49083b6a-dda9-400e-88a4-4f64aff54cd8 service nova] Creating event network-changed:73f87b73-9d7c-43a7-9bd2-dc7fb5450112 for instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a on hv-cinder-84336 May 28 03:40:28.042570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-b5f59ab7-8d6d-4e70-97c6-4a4b3a79b647 req-49083b6a-dda9-400e-88a4-4f64aff54cd8 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046405 May 28 03:40:28.044039 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 321/655] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:28.161894 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8070749d-bd36-450a-b7d5-2baf522ec5e4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:28.170431 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8070749d-bd36-450a-b7d5-2baf522ec5e4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.195357 May 28 03:40:28.171005 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 335/656] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:27 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:28.610753 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-78e2d7d1-69fd-4b97-b762-753f6f4ea47d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:28.611358 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-78e2d7d1-69fd-4b97-b762-753f6f4ea47d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:28.618284 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78e2d7d1-69fd-4b97-b762-753f6f4ea47d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:28.618725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78e2d7d1-69fd-4b97-b762-753f6f4ea47d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:28.619474 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78e2d7d1-69fd-4b97-b762-753f6f4ea47d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:28.620408 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78e2d7d1-69fd-4b97-b762-753f6f4ea47d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:28.755554 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-78e2d7d1-69fd-4b97-b762-753f6f4ea47d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:28.764019 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-78e2d7d1-69fd-4b97-b762-753f6f4ea47d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:28.765535 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-78e2d7d1-69fd-4b97-b762-753f6f4ea47d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:28.766631 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-78e2d7d1-69fd-4b97-b762-753f6f4ea47d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.160179 May 28 03:40:28.767190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 322/657] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:28 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:29.187557 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9d92b564-7b30-410d-b027-6bd0c297c3c5 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:29.188106 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9d92b564-7b30-410d-b027-6bd0c297c3c5 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:29.194543 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9d92b564-7b30-410d-b027-6bd0c297c3c5 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:29.194988 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9d92b564-7b30-410d-b027-6bd0c297c3c5 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:29.195743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9d92b564-7b30-410d-b027-6bd0c297c3c5 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:29.196176 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9d92b564-7b30-410d-b027-6bd0c297c3c5 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:29.319703 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9d92b564-7b30-410d-b027-6bd0c297c3c5 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:29.328021 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9d92b564-7b30-410d-b027-6bd0c297c3c5 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.144351 May 28 03:40:29.328599 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 336/658] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:29 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:29.783567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e4c5c0c3-9e66-41fd-9374-624b1f0faabd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:29.784172 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e4c5c0c3-9e66-41fd-9374-624b1f0faabd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:29.790618 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4c5c0c3-9e66-41fd-9374-624b1f0faabd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:29.791065 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4c5c0c3-9e66-41fd-9374-624b1f0faabd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:29.791821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4c5c0c3-9e66-41fd-9374-624b1f0faabd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:29.792251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e4c5c0c3-9e66-41fd-9374-624b1f0faabd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:29.979899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e4c5c0c3-9e66-41fd-9374-624b1f0faabd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:29.991860 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e4c5c0c3-9e66-41fd-9374-624b1f0faabd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:29.994550 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e4c5c0c3-9e66-41fd-9374-624b1f0faabd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:29.999504 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e4c5c0c3-9e66-41fd-9374-624b1f0faabd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.217019 May 28 03:40:29.999504 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 323/659] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:29 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:30.346202 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d2f49c81-207a-405b-8c1b-1098ac416836 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:30.347622 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d2f49c81-207a-405b-8c1b-1098ac416836 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:30.353558 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2f49c81-207a-405b-8c1b-1098ac416836 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:30.354013 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2f49c81-207a-405b-8c1b-1098ac416836 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:30.355415 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2f49c81-207a-405b-8c1b-1098ac416836 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:30.355860 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2f49c81-207a-405b-8c1b-1098ac416836 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:30.493173 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d2f49c81-207a-405b-8c1b-1098ac416836 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:30.501086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d2f49c81-207a-405b-8c1b-1098ac416836 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.159167 May 28 03:40:30.501682 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 337/660] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:30 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:31.017657 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6a6f45c7-260e-4768-9486-446b050c66bf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:31.018267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6a6f45c7-260e-4768-9486-446b050c66bf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:31.024498 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6a6f45c7-260e-4768-9486-446b050c66bf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:31.024880 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6a6f45c7-260e-4768-9486-446b050c66bf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:31.025661 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6a6f45c7-260e-4768-9486-446b050c66bf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:31.026103 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6a6f45c7-260e-4768-9486-446b050c66bf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:31.110080 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3f93316f-8b3f-44e0-a24f-321594733801 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:31.111396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3f93316f-8b3f-44e0-a24f-321594733801 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:31.118228 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3f93316f-8b3f-44e0-a24f-321594733801 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:31.118678 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3f93316f-8b3f-44e0-a24f-321594733801 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:31.119454 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3f93316f-8b3f-44e0-a24f-321594733801 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:31.120514 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3f93316f-8b3f-44e0-a24f-321594733801 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:31.154028 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3f93316f-8b3f-44e0-a24f-321594733801 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047748 May 28 03:40:31.154830 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 338/661] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:40:31 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) May 28 03:40:31.169993 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6a6f45c7-260e-4768-9486-446b050c66bf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:31.179174 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6a6f45c7-260e-4768-9486-446b050c66bf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:31.180671 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6a6f45c7-260e-4768-9486-446b050c66bf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:31.181844 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6a6f45c7-260e-4768-9486-446b050c66bf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.168013 May 28 03:40:31.182955 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 324/662] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:31 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:31.218389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b04f6c79-cfb7-4316-8cec-95ccb0408d40 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:31.218896 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b04f6c79-cfb7-4316-8cec-95ccb0408d40 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:31.224812 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b04f6c79-cfb7-4316-8cec-95ccb0408d40 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:31.225302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b04f6c79-cfb7-4316-8cec-95ccb0408d40 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:31.226075 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b04f6c79-cfb7-4316-8cec-95ccb0408d40 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:31.226520 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b04f6c79-cfb7-4316-8cec-95ccb0408d40 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:31.251910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b04f6c79-cfb7-4316-8cec-95ccb0408d40 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Going to try to terminate instance {{(pid=124604) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:40:31.519309 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0f3dfda0-f741-483c-b354-f157184fa0a6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:31.519976 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0f3dfda0-f741-483c-b354-f157184fa0a6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:31.526085 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0f3dfda0-f741-483c-b354-f157184fa0a6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:31.526530 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0f3dfda0-f741-483c-b354-f157184fa0a6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:31.527385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0f3dfda0-f741-483c-b354-f157184fa0a6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:31.527856 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0f3dfda0-f741-483c-b354-f157184fa0a6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:31.540085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b04f6c79-cfb7-4316-8cec-95ccb0408d40 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "DELETE /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 204 len: 0 microversion: 2.1 time: 0.325212 May 28 03:40:31.540848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 339/663] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:40:31 2022] DELETE /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:40:31.546275 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=100 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:31.549151 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-55247a1d-5c47-4887-82cf-420b3940c269 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:31.549878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-55247a1d-5c47-4887-82cf-420b3940c269 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:31.555857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-55247a1d-5c47-4887-82cf-420b3940c269 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:31.556253 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-55247a1d-5c47-4887-82cf-420b3940c269 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:31.557022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-55247a1d-5c47-4887-82cf-420b3940c269 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:31.557999 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-55247a1d-5c47-4887-82cf-420b3940c269 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:31.668042 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0f3dfda0-f741-483c-b354-f157184fa0a6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:31.677180 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0f3dfda0-f741-483c-b354-f157184fa0a6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.161734 May 28 03:40:31.677608 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 325/664] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:31 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:31.708272 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-55247a1d-5c47-4887-82cf-420b3940c269 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:31.716317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-55247a1d-5c47-4887-82cf-420b3940c269 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1673 microversion: 2.1 time: 0.171133 May 28 03:40:31.716888 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 340/665] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:31 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1673 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:32.200426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-208c2595-a717-4723-bc63-fb47228c2451 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:32.200944 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-208c2595-a717-4723-bc63-fb47228c2451 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:32.207898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-208c2595-a717-4723-bc63-fb47228c2451 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:32.208357 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-208c2595-a717-4723-bc63-fb47228c2451 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:32.209144 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-208c2595-a717-4723-bc63-fb47228c2451 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:32.209586 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-208c2595-a717-4723-bc63-fb47228c2451 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:32.338396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-208c2595-a717-4723-bc63-fb47228c2451 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:32.346300 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-208c2595-a717-4723-bc63-fb47228c2451 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:32.347866 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-208c2595-a717-4723-bc63-fb47228c2451 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:32.348972 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-208c2595-a717-4723-bc63-fb47228c2451 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.153132 May 28 03:40:32.349560 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 326/666] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:32 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:32.693725 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f34bfbe3-1545-424f-a23d-643786c0786b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:32.694330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f34bfbe3-1545-424f-a23d-643786c0786b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:32.701836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f34bfbe3-1545-424f-a23d-643786c0786b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:32.702288 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f34bfbe3-1545-424f-a23d-643786c0786b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:32.703043 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f34bfbe3-1545-424f-a23d-643786c0786b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:32.703474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f34bfbe3-1545-424f-a23d-643786c0786b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:32.739090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f9d1bd00-f011-474d-9b59-d3279e2b242e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:32.739702 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f9d1bd00-f011-474d-9b59-d3279e2b242e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:32.746761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f9d1bd00-f011-474d-9b59-d3279e2b242e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:32.747217 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f9d1bd00-f011-474d-9b59-d3279e2b242e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:32.747977 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f9d1bd00-f011-474d-9b59-d3279e2b242e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:32.748593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f9d1bd00-f011-474d-9b59-d3279e2b242e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:32.837820 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f34bfbe3-1545-424f-a23d-643786c0786b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:32.855774 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f34bfbe3-1545-424f-a23d-643786c0786b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.165784 May 28 03:40:32.856212 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 341/667] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:32 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:32.887956 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f9d1bd00-f011-474d-9b59-d3279e2b242e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:32.893582 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f9d1bd00-f011-474d-9b59-d3279e2b242e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1673 microversion: 2.1 time: 0.164245 May 28 03:40:32.894148 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 327/668] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:32 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1673 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:33.367089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b9c0ed03-b94d-4638-98cc-29fcc520bb9f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:33.367702 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b9c0ed03-b94d-4638-98cc-29fcc520bb9f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:33.374310 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b9c0ed03-b94d-4638-98cc-29fcc520bb9f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:33.374755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b9c0ed03-b94d-4638-98cc-29fcc520bb9f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:33.375526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b9c0ed03-b94d-4638-98cc-29fcc520bb9f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:33.375973 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b9c0ed03-b94d-4638-98cc-29fcc520bb9f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:33.508183 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b9c0ed03-b94d-4638-98cc-29fcc520bb9f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:33.516614 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b9c0ed03-b94d-4638-98cc-29fcc520bb9f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:33.518142 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b9c0ed03-b94d-4638-98cc-29fcc520bb9f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:33.519241 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b9c0ed03-b94d-4638-98cc-29fcc520bb9f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.156276 May 28 03:40:33.519811 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 342/669] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:33 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:33.873753 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4acf0ad4-9632-4375-a178-201199d1a84c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:33.874312 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4acf0ad4-9632-4375-a178-201199d1a84c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:33.880818 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4acf0ad4-9632-4375-a178-201199d1a84c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:33.881319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4acf0ad4-9632-4375-a178-201199d1a84c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:33.882112 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4acf0ad4-9632-4375-a178-201199d1a84c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:33.882503 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4acf0ad4-9632-4375-a178-201199d1a84c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:33.899161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:33.910454 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-eed1ccc1-4673-4d9b-8cc3-825a1f797c04 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:33.912426 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-eed1ccc1-4673-4d9b-8cc3-825a1f797c04 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:33.917343 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-eed1ccc1-4673-4d9b-8cc3-825a1f797c04 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:33.918027 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-eed1ccc1-4673-4d9b-8cc3-825a1f797c04 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:33.918803 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-eed1ccc1-4673-4d9b-8cc3-825a1f797c04 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:33.919237 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-eed1ccc1-4673-4d9b-8cc3-825a1f797c04 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:34.032788 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4acf0ad4-9632-4375-a178-201199d1a84c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:34.041821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4acf0ad4-9632-4375-a178-201199d1a84c tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.172371 May 28 03:40:34.042396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 328/670] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:33 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:34.054854 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-eed1ccc1-4673-4d9b-8cc3-825a1f797c04 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:34.063017 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-eed1ccc1-4673-4d9b-8cc3-825a1f797c04 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1673 microversion: 2.1 time: 0.156348 May 28 03:40:34.063621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 343/671] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:33 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1673 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:34.537758 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-daf1ad64-b2ef-449b-b790-0808df7088e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:34.538354 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-daf1ad64-b2ef-449b-b790-0808df7088e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:34.540177 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=12 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:34.545576 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-daf1ad64-b2ef-449b-b790-0808df7088e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:34.546016 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-daf1ad64-b2ef-449b-b790-0808df7088e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:34.546762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-daf1ad64-b2ef-449b-b790-0808df7088e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:34.547190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-daf1ad64-b2ef-449b-b790-0808df7088e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:34.687203 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-daf1ad64-b2ef-449b-b790-0808df7088e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:34.695199 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-daf1ad64-b2ef-449b-b790-0808df7088e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:34.697033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-daf1ad64-b2ef-449b-b790-0808df7088e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:34.698804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-daf1ad64-b2ef-449b-b790-0808df7088e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.164335 May 28 03:40:34.699372 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 329/672] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:34 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:35.058872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-928315e7-a487-4d52-931a-22b681c3123e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:35.059555 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-928315e7-a487-4d52-931a-22b681c3123e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:35.066140 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-928315e7-a487-4d52-931a-22b681c3123e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:35.066585 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-928315e7-a487-4d52-931a-22b681c3123e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:35.067348 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-928315e7-a487-4d52-931a-22b681c3123e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:35.067776 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-928315e7-a487-4d52-931a-22b681c3123e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:35.080763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-777dafe5-688e-4520-b451-566f1b9cd025 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:35.081428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-777dafe5-688e-4520-b451-566f1b9cd025 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:35.090339 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-777dafe5-688e-4520-b451-566f1b9cd025 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:35.090998 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-777dafe5-688e-4520-b451-566f1b9cd025 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:35.091988 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-777dafe5-688e-4520-b451-566f1b9cd025 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:35.092607 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-777dafe5-688e-4520-b451-566f1b9cd025 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:35.206677 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-928315e7-a487-4d52-931a-22b681c3123e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:35.210660 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:35.216289 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-928315e7-a487-4d52-931a-22b681c3123e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.161231 May 28 03:40:35.217147 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 344/673] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:35 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:35.298051 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-777dafe5-688e-4520-b451-566f1b9cd025 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:35.305513 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-777dafe5-688e-4520-b451-566f1b9cd025 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1673 microversion: 2.1 time: 0.229244 May 28 03:40:35.306097 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 330/674] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:35 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1673 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:35.604958 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-ca0af838-da2e-4628-8494-07de36b30c40 req-520bf863-6465-4be9-a349-c5c5d3d08ee1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6be2aeaa-8d90-4e51-87d2-d02a8566ac1a", "name": "network-vif-plugged", "status": "completed", "tag": "73f87b73-9d7c-43a7-9bd2-dc7fb5450112"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:35.617760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-ca0af838-da2e-4628-8494-07de36b30c40 req-520bf863-6465-4be9-a349-c5c5d3d08ee1 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:35.623371 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-ca0af838-da2e-4628-8494-07de36b30c40 req-520bf863-6465-4be9-a349-c5c5d3d08ee1 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:35.633600 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-ca0af838-da2e-4628-8494-07de36b30c40 req-520bf863-6465-4be9-a349-c5c5d3d08ee1 service nova] Creating event network-vif-plugged:73f87b73-9d7c-43a7-9bd2-dc7fb5450112 for instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a on hv-cinder-84336 May 28 03:40:35.639349 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-ca0af838-da2e-4628-8494-07de36b30c40 req-520bf863-6465-4be9-a349-c5c5d3d08ee1 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042351 May 28 03:40:35.640583 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 345/675] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:35.717040 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7d5aa5fb-cfda-4193-9703-09065050e1a0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:35.717934 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7d5aa5fb-cfda-4193-9703-09065050e1a0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:35.728407 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d5aa5fb-cfda-4193-9703-09065050e1a0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:35.729430 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-83d6d2c6-cd42-46e1-8c75-c79b43aaa334 req-4edeabda-bc61-46c0-aa6b-b51fe6eaf190 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "da053642-eac6-4484-9557-f0b878712cf2", "name": "network-vif-deleted", "tag": "63e23069-fb2c-4804-8da1-bf97b736bc08"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:35.730128 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d5aa5fb-cfda-4193-9703-09065050e1a0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:35.732821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d5aa5fb-cfda-4193-9703-09065050e1a0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:35.732821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7d5aa5fb-cfda-4193-9703-09065050e1a0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:35.752560 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-83d6d2c6-cd42-46e1-8c75-c79b43aaa334 req-4edeabda-bc61-46c0-aa6b-b51fe6eaf190 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:35.753134 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-83d6d2c6-cd42-46e1-8c75-c79b43aaa334 req-4edeabda-bc61-46c0-aa6b-b51fe6eaf190 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:35.788760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-83d6d2c6-cd42-46e1-8c75-c79b43aaa334 req-4edeabda-bc61-46c0-aa6b-b51fe6eaf190 service nova] Creating event network-vif-deleted:63e23069-fb2c-4804-8da1-bf97b736bc08 for instance da053642-eac6-4484-9557-f0b878712cf2 on hv-cinder-84336 May 28 03:40:35.799458 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-83d6d2c6-cd42-46e1-8c75-c79b43aaa334 req-4edeabda-bc61-46c0-aa6b-b51fe6eaf190 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075463 May 28 03:40:35.801089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 346/676] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:35.935018 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7d5aa5fb-cfda-4193-9703-09065050e1a0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:35.945046 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7d5aa5fb-cfda-4193-9703-09065050e1a0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:35.946902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7d5aa5fb-cfda-4193-9703-09065050e1a0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:35.948675 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7d5aa5fb-cfda-4193-9703-09065050e1a0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.236164 May 28 03:40:35.959245 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 331/677] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:35 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:36.234143 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5a5bf9ce-3f48-4e1d-8cb0-cf8f4ccbf1ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:36.235383 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5a5bf9ce-3f48-4e1d-8cb0-cf8f4ccbf1ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:36.242597 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5a5bf9ce-3f48-4e1d-8cb0-cf8f4ccbf1ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:36.243047 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5a5bf9ce-3f48-4e1d-8cb0-cf8f4ccbf1ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:36.244034 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5a5bf9ce-3f48-4e1d-8cb0-cf8f4ccbf1ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:36.244535 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5a5bf9ce-3f48-4e1d-8cb0-cf8f4ccbf1ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:36.324033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7079451f-4bb2-4caa-80ed-2cda82107697 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:36.325797 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7079451f-4bb2-4caa-80ed-2cda82107697 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:36.333800 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7079451f-4bb2-4caa-80ed-2cda82107697 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:36.334531 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7079451f-4bb2-4caa-80ed-2cda82107697 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:36.336970 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7079451f-4bb2-4caa-80ed-2cda82107697 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:36.336970 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7079451f-4bb2-4caa-80ed-2cda82107697 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:36.376288 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7079451f-4bb2-4caa-80ed-2cda82107697 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:36.376733 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7079451f-4bb2-4caa-80ed-2cda82107697 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:36.377213 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-7079451f-4bb2-4caa-80ed-2cda82107697 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lazy-loading 'fault' on Instance uuid da053642-eac6-4484-9557-f0b878712cf2 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:40:36.393528 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5a5bf9ce-3f48-4e1d-8cb0-cf8f4ccbf1ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:36.401685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5a5bf9ce-3f48-4e1d-8cb0-cf8f4ccbf1ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.171606 May 28 03:40:36.402597 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 347/678] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:36 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:36.426286 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7079451f-4bb2-4caa-80ed-2cda82107697 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:36.433744 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7079451f-4bb2-4caa-80ed-2cda82107697 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1327 microversion: 2.1 time: 0.114218 May 28 03:40:36.434236 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 332/679] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:36 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1327 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:36.967485 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-66ba536c-8d11-439f-a61d-0597081086d7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:36.968022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-66ba536c-8d11-439f-a61d-0597081086d7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:36.974104 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-66ba536c-8d11-439f-a61d-0597081086d7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:36.974547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-66ba536c-8d11-439f-a61d-0597081086d7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:36.975326 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-66ba536c-8d11-439f-a61d-0597081086d7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:36.975754 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-66ba536c-8d11-439f-a61d-0597081086d7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:37.261845 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-66ba536c-8d11-439f-a61d-0597081086d7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:37.269670 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-66ba536c-8d11-439f-a61d-0597081086d7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:37.271173 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-66ba536c-8d11-439f-a61d-0597081086d7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:37.272275 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-66ba536c-8d11-439f-a61d-0597081086d7 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.309316 May 28 03:40:37.272840 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 348/680] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:36 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:37.418934 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0c14af19-94d7-4964-a7de-eb6db8e91709 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:37.419540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0c14af19-94d7-4964-a7de-eb6db8e91709 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:37.426015 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c14af19-94d7-4964-a7de-eb6db8e91709 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:37.426454 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c14af19-94d7-4964-a7de-eb6db8e91709 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:37.427237 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c14af19-94d7-4964-a7de-eb6db8e91709 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:37.427667 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c14af19-94d7-4964-a7de-eb6db8e91709 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:37.451773 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0eb66839-5926-4eaa-9ed6-e3893d5944f1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:37.452388 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0eb66839-5926-4eaa-9ed6-e3893d5944f1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:37.458354 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0eb66839-5926-4eaa-9ed6-e3893d5944f1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:37.458809 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0eb66839-5926-4eaa-9ed6-e3893d5944f1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:37.459582 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0eb66839-5926-4eaa-9ed6-e3893d5944f1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:37.460022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0eb66839-5926-4eaa-9ed6-e3893d5944f1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:37.490934 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0eb66839-5926-4eaa-9ed6-e3893d5944f1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:37.491385 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0eb66839-5926-4eaa-9ed6-e3893d5944f1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:37.492092 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-0eb66839-5926-4eaa-9ed6-e3893d5944f1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lazy-loading 'fault' on Instance uuid da053642-eac6-4484-9557-f0b878712cf2 {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:40:37.542745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0eb66839-5926-4eaa-9ed6-e3893d5944f1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:37.550059 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0eb66839-5926-4eaa-9ed6-e3893d5944f1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1327 microversion: 2.1 time: 0.102653 May 28 03:40:37.550933 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 349/681] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:37 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1327 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:37.556705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0c14af19-94d7-4964-a7de-eb6db8e91709 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:37.565134 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0c14af19-94d7-4964-a7de-eb6db8e91709 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.150004 May 28 03:40:37.565708 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 333/682] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:37 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:38.292045 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-abb14661-55c1-41f5-bdcd-6aef2ebff315 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:38.292646 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-abb14661-55c1-41f5-bdcd-6aef2ebff315 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:38.299386 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-abb14661-55c1-41f5-bdcd-6aef2ebff315 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:38.299838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-abb14661-55c1-41f5-bdcd-6aef2ebff315 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:38.300590 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-abb14661-55c1-41f5-bdcd-6aef2ebff315 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:38.301018 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-abb14661-55c1-41f5-bdcd-6aef2ebff315 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:38.432115 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-abb14661-55c1-41f5-bdcd-6aef2ebff315 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:38.441531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-abb14661-55c1-41f5-bdcd-6aef2ebff315 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:38.443194 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-abb14661-55c1-41f5-bdcd-6aef2ebff315 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:38.444354 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-abb14661-55c1-41f5-bdcd-6aef2ebff315 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.156301 May 28 03:40:38.444897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 350/683] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:38 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:38.567747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-eef41180-0fab-4684-808d-bf868ddfc329 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:38.568302 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-eef41180-0fab-4684-808d-bf868ddfc329 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:38.574905 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eef41180-0fab-4684-808d-bf868ddfc329 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:38.575343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eef41180-0fab-4684-808d-bf868ddfc329 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:38.576090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eef41180-0fab-4684-808d-bf868ddfc329 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:38.576522 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eef41180-0fab-4684-808d-bf868ddfc329 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:38.582593 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c78823fc-3567-41a5-b3c7-5d95b01195e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:38.583229 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c78823fc-3567-41a5-b3c7-5d95b01195e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:38.589313 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c78823fc-3567-41a5-b3c7-5d95b01195e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:38.589765 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c78823fc-3567-41a5-b3c7-5d95b01195e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:38.590541 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c78823fc-3567-41a5-b3c7-5d95b01195e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:38.590978 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c78823fc-3567-41a5-b3c7-5d95b01195e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:38.613961 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eef41180-0fab-4684-808d-bf868ddfc329 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:38.614400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-eef41180-0fab-4684-808d-bf868ddfc329 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:38.614872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-eef41180-0fab-4684-808d-bf868ddfc329 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lazy-loading 'fault' on Instance uuid da053642-eac6-4484-9557-f0b878712cf2 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:40:38.657476 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-eef41180-0fab-4684-808d-bf868ddfc329 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:38.664751 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-eef41180-0fab-4684-808d-bf868ddfc329 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1327 microversion: 2.1 time: 0.101006 May 28 03:40:38.665347 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 334/684] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:38 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1327 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:38.723879 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c78823fc-3567-41a5-b3c7-5d95b01195e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:38.732562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c78823fc-3567-41a5-b3c7-5d95b01195e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.153781 May 28 03:40:38.733227 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 351/685] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:38 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:39.467552 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-34da4acd-48c3-49eb-8b8b-ba66892a6711 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:39.468064 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-34da4acd-48c3-49eb-8b8b-ba66892a6711 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:39.474316 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-34da4acd-48c3-49eb-8b8b-ba66892a6711 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:39.474760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-34da4acd-48c3-49eb-8b8b-ba66892a6711 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:39.475522 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-34da4acd-48c3-49eb-8b8b-ba66892a6711 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:39.475955 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-34da4acd-48c3-49eb-8b8b-ba66892a6711 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:39.604019 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-34da4acd-48c3-49eb-8b8b-ba66892a6711 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:39.612069 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-34da4acd-48c3-49eb-8b8b-ba66892a6711 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:39.613638 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-34da4acd-48c3-49eb-8b8b-ba66892a6711 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:39.614741 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-34da4acd-48c3-49eb-8b8b-ba66892a6711 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.151770 May 28 03:40:39.615290 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 335/686] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:39 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:39.682064 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c4e4a94f-9dfc-4977-8f05-1c2d4466cdd1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:39.682662 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c4e4a94f-9dfc-4977-8f05-1c2d4466cdd1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:39.689161 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4e4a94f-9dfc-4977-8f05-1c2d4466cdd1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:39.689616 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4e4a94f-9dfc-4977-8f05-1c2d4466cdd1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:39.690380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4e4a94f-9dfc-4977-8f05-1c2d4466cdd1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:39.690843 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4e4a94f-9dfc-4977-8f05-1c2d4466cdd1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:39.721666 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4e4a94f-9dfc-4977-8f05-1c2d4466cdd1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:39.722005 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4e4a94f-9dfc-4977-8f05-1c2d4466cdd1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:39.722469 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-c4e4a94f-9dfc-4977-8f05-1c2d4466cdd1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lazy-loading 'fault' on Instance uuid da053642-eac6-4484-9557-f0b878712cf2 {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:40:39.751044 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-39cc3e48-5526-4ca1-b0c3-b5556ad08198 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:39.751650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-39cc3e48-5526-4ca1-b0c3-b5556ad08198 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:39.758072 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-39cc3e48-5526-4ca1-b0c3-b5556ad08198 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:39.758523 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-39cc3e48-5526-4ca1-b0c3-b5556ad08198 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:39.759351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-39cc3e48-5526-4ca1-b0c3-b5556ad08198 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:39.759787 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-39cc3e48-5526-4ca1-b0c3-b5556ad08198 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:39.766872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c4e4a94f-9dfc-4977-8f05-1c2d4466cdd1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:39.775040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c4e4a94f-9dfc-4977-8f05-1c2d4466cdd1 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 200 len: 1327 microversion: 2.1 time: 0.096378 May 28 03:40:39.775731 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 352/687] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:39 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 1327 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:39.894114 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-39cc3e48-5526-4ca1-b0c3-b5556ad08198 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:39.902277 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-39cc3e48-5526-4ca1-b0c3-b5556ad08198 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.155031 May 28 03:40:39.912974 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 336/688] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:39 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:40.632513 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-adc6db6d-9133-4900-868b-e133deeac617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:40.633039 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-adc6db6d-9133-4900-868b-e133deeac617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:40.639781 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-adc6db6d-9133-4900-868b-e133deeac617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:40.640262 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-adc6db6d-9133-4900-868b-e133deeac617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:40.641005 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-adc6db6d-9133-4900-868b-e133deeac617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:40.641473 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-adc6db6d-9133-4900-868b-e133deeac617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:40.792157 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f15d279b-0db1-4bb8-aa24-49de7c5ea66e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:40.793470 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f15d279b-0db1-4bb8-aa24-49de7c5ea66e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:40.793926 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-adc6db6d-9133-4900-868b-e133deeac617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:40.801073 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f15d279b-0db1-4bb8-aa24-49de7c5ea66e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:40.804537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f15d279b-0db1-4bb8-aa24-49de7c5ea66e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:40.804537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f15d279b-0db1-4bb8-aa24-49de7c5ea66e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:40.804740 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-adc6db6d-9133-4900-868b-e133deeac617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:40.804856 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f15d279b-0db1-4bb8-aa24-49de7c5ea66e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:40.804934 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-adc6db6d-9133-4900-868b-e133deeac617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:40.805028 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-adc6db6d-9133-4900-868b-e133deeac617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1406 microversion: 2.20 time: 0.176659 May 28 03:40:40.805637 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 353/689] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:40 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1406 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:40.819602 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-f15d279b-0db1-4bb8-aa24-49de7c5ea66e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] HTTP exception thrown: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f15d279b-0db1-4bb8-aa24-49de7c5ea66e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Request method failure captured: May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 HTTP/1.1 May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZiiQypFQQ1bdISGQo9C2YguYEEU-tBqQGVthYz_izW7K5gHqUD5gE5Vtsn9MQmMxa3fO2B2uT4EMCIaBoWDOcTCa_6QL577b2i5ZTpp-2FFBn6eD7fm9RJIlmaKR4ZpoEyCHy7RVgamveJZw6aiAUZ4N9PvH67OUDmLYqMGBctoWpU May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:40:40.821272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeTestJSON-1322958216 May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: dc6a3335562448d1934fa18e8f51b119 May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:40:40.822862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': 'da053642-eac6-4484-9557-f0b878712cf2'} May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:40:40.824455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:40:40.825902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:40.827143 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f15d279b-0db1-4bb8-aa24-49de7c5ea66e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Returning 404 to user: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:40:40.827143 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f15d279b-0db1-4bb8-aa24-49de7c5ea66e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 404 len: 111 microversion: 2.1 time: 0.034749 May 28 03:40:40.827143 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 337/690] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:40 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:40:40.832363 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2fa89bba-e142-4b8e-919b-1105f98409cb tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '.version_select at 0x7feb1f5884c0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:40.920208 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1dabf23c-e6cc-431d-bebb-224f54bd13a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:40.920817 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1dabf23c-e6cc-431d-bebb-224f54bd13a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:40.927294 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dabf23c-e6cc-431d-bebb-224f54bd13a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:40.927739 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dabf23c-e6cc-431d-bebb-224f54bd13a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:40.928499 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dabf23c-e6cc-431d-bebb-224f54bd13a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:40.928932 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dabf23c-e6cc-431d-bebb-224f54bd13a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:40.948911 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2fa89bba-e142-4b8e-919b-1105f98409cb tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-447830529" status: 202 len: 0 microversion: 2.1 time: 0.119647 May 28 03:40:40.953550 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 354/691] 10.222.0.41 () {58 vars in 1217 bytes} [Sat May 28 03:40:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-447830529 => generated 0 bytes in 125 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 28 03:40:41.075940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1dabf23c-e6cc-431d-bebb-224f54bd13a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:41.083340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1dabf23c-e6cc-431d-bebb-224f54bd13a1 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.167327 May 28 03:40:41.084730 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 338/692] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:40 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:41.824678 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4c5ca276-d47b-45a0-a902-698d8ce392a1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:41.825323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4c5ca276-d47b-45a0-a902-698d8ce392a1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:41.831520 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c5ca276-d47b-45a0-a902-698d8ce392a1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:41.831993 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c5ca276-d47b-45a0-a902-698d8ce392a1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:41.832764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c5ca276-d47b-45a0-a902-698d8ce392a1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:41.833237 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c5ca276-d47b-45a0-a902-698d8ce392a1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:41.940103 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ca2a471e-81f5-44c1-b43a-66b8fb2af854 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Action: 'create', calling method: .version_select at 0x7feb1fd20790>, body: {"keypair": {"name": "tempest-keypair-107055024", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPBcroAuzyGrxtNYrU0hixWTuqfnqswiWxprkBSM/PlQxcd5LNN2FlRmCPip98gf2fsJSXLpTcS+cv76Y4jIMT7ObB9eCXtYLxHPff7qaZYo8F9+HRkWlHPIT9N0bltflw=="}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:41.956719 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-ca2a471e-81f5-44c1-b43a-66b8fb2af854 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Getting quotas for project 70eaf602c8db4ec1bfb26e5eb3b13c82. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:40:41.960660 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-ca2a471e-81f5-44c1-b43a-66b8fb2af854 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Getting quotas for user dc6a3335562448d1934fa18e8f51b119 and project 70eaf602c8db4ec1bfb26e5eb3b13c82. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:40:41.987256 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4c5ca276-d47b-45a0-a902-698d8ce392a1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:41.996074 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4c5ca276-d47b-45a0-a902-698d8ce392a1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:41.997625 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4c5ca276-d47b-45a0-a902-698d8ce392a1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:41.998783 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4c5ca276-d47b-45a0-a902-698d8ce392a1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1599 microversion: 2.20 time: 0.178632 May 28 03:40:41.999336 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 355/693] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:40:41 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1599 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:40:42.016113 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d5151624-a978-4a39-b66f-a94fff3137cd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:42.016614 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d5151624-a978-4a39-b66f-a94fff3137cd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:42.022602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5151624-a978-4a39-b66f-a94fff3137cd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:42.023152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5151624-a978-4a39-b66f-a94fff3137cd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:42.023822 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5151624-a978-4a39-b66f-a94fff3137cd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:42.024276 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5151624-a978-4a39-b66f-a94fff3137cd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:42.074661 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ca2a471e-81f5-44c1-b43a-66b8fb2af854 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.138521 May 28 03:40:42.075801 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 339/694] 10.222.0.41 () {60 vars in 1158 bytes} [Sat May 28 03:40:41 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:40:42.106468 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d5151624-a978-4a39-b66f-a94fff3137cd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.093816 May 28 03:40:42.107283 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 356/695] 10.222.0.41 () {60 vars in 1315 bytes} [Sat May 28 03:40:42 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/os-interface => generated 285 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 28 03:40:42.107423 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-fc5a92e4-fbd7-4a05-9afe-70fb77f4e174 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:42.107821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-fc5a92e4-fbd7-4a05-9afe-70fb77f4e174 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:42.114376 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fc5a92e4-fbd7-4a05-9afe-70fb77f4e174 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:42.114827 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fc5a92e4-fbd7-4a05-9afe-70fb77f4e174 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:42.121426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fc5a92e4-fbd7-4a05-9afe-70fb77f4e174 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:42.121426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fc5a92e4-fbd7-4a05-9afe-70fb77f4e174 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:42.274164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-fc5a92e4-fbd7-4a05-9afe-70fb77f4e174 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:42.282452 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-fc5a92e4-fbd7-4a05-9afe-70fb77f4e174 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.184115 May 28 03:40:42.283133 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 340/696] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:42 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:43.299975 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5b7fd96e-1ebb-4868-9d1a-b97b4241e35e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:43.301322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5b7fd96e-1ebb-4868-9d1a-b97b4241e35e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:43.309001 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5b7fd96e-1ebb-4868-9d1a-b97b4241e35e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:43.309740 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5b7fd96e-1ebb-4868-9d1a-b97b4241e35e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:43.319492 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5b7fd96e-1ebb-4868-9d1a-b97b4241e35e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:43.319492 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5b7fd96e-1ebb-4868-9d1a-b97b4241e35e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:43.365460 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-4ff616d8-8cda-495b-974d-5c5f5dff456b req-0494a5e4-1989-4062-bc77-e2d451049493 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6be2aeaa-8d90-4e51-87d2-d02a8566ac1a", "tag": "73f87b73-9d7c-43a7-9bd2-dc7fb5450112"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:43.377443 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-4ff616d8-8cda-495b-974d-5c5f5dff456b req-0494a5e4-1989-4062-bc77-e2d451049493 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:43.378247 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-4ff616d8-8cda-495b-974d-5c5f5dff456b req-0494a5e4-1989-4062-bc77-e2d451049493 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:43.391451 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-4ff616d8-8cda-495b-974d-5c5f5dff456b req-0494a5e4-1989-4062-bc77-e2d451049493 service nova] Creating event network-changed:73f87b73-9d7c-43a7-9bd2-dc7fb5450112 for instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a on hv-cinder-84336 May 28 03:40:43.399474 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-4ff616d8-8cda-495b-974d-5c5f5dff456b req-0494a5e4-1989-4062-bc77-e2d451049493 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038364 May 28 03:40:43.400411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 341/697] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:40:43.482114 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5b7fd96e-1ebb-4868-9d1a-b97b4241e35e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:43.496939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5b7fd96e-1ebb-4868-9d1a-b97b4241e35e tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.200602 May 28 03:40:43.498284 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 357/698] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:43 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:44.516475 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-559fdd77-9f05-491d-a6be-36db1656b88a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:44.517253 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-559fdd77-9f05-491d-a6be-36db1656b88a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:44.523463 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-559fdd77-9f05-491d-a6be-36db1656b88a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:44.523909 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-559fdd77-9f05-491d-a6be-36db1656b88a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:44.524671 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-559fdd77-9f05-491d-a6be-36db1656b88a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:44.525125 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-559fdd77-9f05-491d-a6be-36db1656b88a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:44.667016 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-559fdd77-9f05-491d-a6be-36db1656b88a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:44.675053 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-559fdd77-9f05-491d-a6be-36db1656b88a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.163442 May 28 03:40:44.675622 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 342/699] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:44 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:44.806960 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-181830618", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "9cd091c1-088f-4b49-b9d5-c93354a4ddb5"}], "security_groups": [{"name": "tempest-securitygroup--378387508"}], "key_name": "tempest-keypair-107055024", "user_data": "}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:44.911809 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] validate_networks() for [('9cd091c1-088f-4b49-b9d5-c93354a4ddb5', None, None, None, None, None)] {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:40:45.286961 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:40:45.287541 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:40:45.287875 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:40:45.288299 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:40:45.288710 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:40:45.297798 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:45.299063 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:45.299063 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:45.299451 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:45.316093 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Getting quotas for project 70eaf602c8db4ec1bfb26e5eb3b13c82. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:40:45.319982 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Getting quotas for user dc6a3335562448d1934fa18e8f51b119 and project 70eaf602c8db4ec1bfb26e5eb3b13c82. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:40:45.331251 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:40:45.333751 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:40:45.458661 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-dc5c69dc-5dae-4193-a981-e79b37d1c2ab tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.656109 May 28 03:40:45.461175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 358/700] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:40:44 2022] POST /compute/v2.1/servers => generated 401 bytes in 658 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:40:45.474704 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-52df9fc9-e314-463e-8d4e-a3c4c53e45ac tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:45.475743 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-52df9fc9-e314-463e-8d4e-a3c4c53e45ac tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:45.479020 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:45.544459 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-52df9fc9-e314-463e-8d4e-a3c4c53e45ac tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:45.551521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-52df9fc9-e314-463e-8d4e-a3c4c53e45ac tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1236 microversion: 2.1 time: 0.084149 May 28 03:40:45.552658 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 343/701] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:45 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1236 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:45.693414 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-805b155d-6a46-4c8c-8ec2-6e3f53c3acf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:45.694038 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-805b155d-6a46-4c8c-8ec2-6e3f53c3acf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:45.701652 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-805b155d-6a46-4c8c-8ec2-6e3f53c3acf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:45.702102 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-805b155d-6a46-4c8c-8ec2-6e3f53c3acf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:45.702904 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-805b155d-6a46-4c8c-8ec2-6e3f53c3acf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:45.703366 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-805b155d-6a46-4c8c-8ec2-6e3f53c3acf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:45.844565 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-805b155d-6a46-4c8c-8ec2-6e3f53c3acf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:45.852803 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-805b155d-6a46-4c8c-8ec2-6e3f53c3acf8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.163484 May 28 03:40:45.853399 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 359/702] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:45 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:46.568833 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8c8eede6-8195-492c-b5f6-2d24478efed4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:46.569583 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8c8eede6-8195-492c-b5f6-2d24478efed4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:46.577575 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c8eede6-8195-492c-b5f6-2d24478efed4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:46.578019 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c8eede6-8195-492c-b5f6-2d24478efed4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:46.578784 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c8eede6-8195-492c-b5f6-2d24478efed4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:46.579217 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8c8eede6-8195-492c-b5f6-2d24478efed4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:46.646306 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8c8eede6-8195-492c-b5f6-2d24478efed4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:46.649620 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=100 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:46.655625 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8c8eede6-8195-492c-b5f6-2d24478efed4 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1240 microversion: 2.1 time: 0.090586 May 28 03:40:46.656194 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 344/703] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:46 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1240 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:46.870259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-78d8c7f2-cdcf-4e01-861e-ecbf3e29d14a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:46.870793 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-78d8c7f2-cdcf-4e01-861e-ecbf3e29d14a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:46.877856 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-78d8c7f2-cdcf-4e01-861e-ecbf3e29d14a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:46.878307 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-78d8c7f2-cdcf-4e01-861e-ecbf3e29d14a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:46.879078 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-78d8c7f2-cdcf-4e01-861e-ecbf3e29d14a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:46.879519 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-78d8c7f2-cdcf-4e01-861e-ecbf3e29d14a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:47.178056 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-78d8c7f2-cdcf-4e01-861e-ecbf3e29d14a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:47.187718 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-78d8c7f2-cdcf-4e01-861e-ecbf3e29d14a tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1735 microversion: 2.1 time: 0.321708 May 28 03:40:47.188286 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 360/704] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:46 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1735 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:47.672802 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b056d028-8fbb-4cb7-a6bf-1718099f839c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:47.673969 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-b056d028-8fbb-4cb7-a6bf-1718099f839c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:47.680321 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b056d028-8fbb-4cb7-a6bf-1718099f839c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:47.680832 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b056d028-8fbb-4cb7-a6bf-1718099f839c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:47.681654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b056d028-8fbb-4cb7-a6bf-1718099f839c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:47.682091 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b056d028-8fbb-4cb7-a6bf-1718099f839c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:47.743147 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-b056d028-8fbb-4cb7-a6bf-1718099f839c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:47.754398 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-b056d028-8fbb-4cb7-a6bf-1718099f839c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1232 microversion: 2.1 time: 0.085367 May 28 03:40:47.756727 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 345/705] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:47 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1232 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:48.205863 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-434eb9df-f0b0-4eab-b0f3-af0cfb0150ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:48.206405 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-434eb9df-f0b0-4eab-b0f3-af0cfb0150ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:48.209986 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:48.213445 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-434eb9df-f0b0-4eab-b0f3-af0cfb0150ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.213899 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-434eb9df-f0b0-4eab-b0f3-af0cfb0150ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:48.214715 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-434eb9df-f0b0-4eab-b0f3-af0cfb0150ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.215151 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-434eb9df-f0b0-4eab-b0f3-af0cfb0150ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:48.319871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0d947f75-1689-45c6-9a4f-0d5d41770292 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:48.321706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0d947f75-1689-45c6-9a4f-0d5d41770292 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:48.336587 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d947f75-1689-45c6-9a4f-0d5d41770292 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.337370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d947f75-1689-45c6-9a4f-0d5d41770292 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:48.338323 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d947f75-1689-45c6-9a4f-0d5d41770292 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.338910 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0d947f75-1689-45c6-9a4f-0d5d41770292 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:48.352174 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-434eb9df-f0b0-4eab-b0f3-af0cfb0150ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:48.362878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-434eb9df-f0b0-4eab-b0f3-af0cfb0150ed tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1715 microversion: 2.1 time: 0.160873 May 28 03:40:48.363652 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 361/706] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:48 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1715 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:48.493412 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-0d947f75-1689-45c6-9a4f-0d5d41770292 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lazy-loading 'services' on Instance uuid 1eaf0b19-b250-4970-adb1-1cdad09fff17 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:40:48.498563 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c44c2750-73c7-4484-83d5-1e982ba52a85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:48.499307 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c44c2750-73c7-4484-83d5-1e982ba52a85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:48.516630 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c44c2750-73c7-4484-83d5-1e982ba52a85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.517077 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c44c2750-73c7-4484-83d5-1e982ba52a85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:48.517925 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c44c2750-73c7-4484-83d5-1e982ba52a85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.518366 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c44c2750-73c7-4484-83d5-1e982ba52a85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:48.625252 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-c44c2750-73c7-4484-83d5-1e982ba52a85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] HTTP exception thrown: Cannot 'detach_volume' instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 while it is in vm_state rescued May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c44c2750-73c7-4484-83d5-1e982ba52a85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Request method failure captured: May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments/3e4c9aa3-01fe-4aaa-bdfd-4cf2851d934c HTTP/1.1 May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZioFSMjchJLpIZqod5q-7lgl8fEY36gE52nylmpmtlBCMGq9sB1-y2xrNX1ntFqjnFW5i92fC2liYIvh0AHl9bCuVQTnAqKQCbialBgg6tn7xuEEa8ZKcsvKvenrdvzcDN3coDLFL7-jztA0Tb3mMeshJqy19Ht6r4OU5cn2edXS_E May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 5cab8255c6134aebb9c93493a2342cb1 May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:40:48.627088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:40:48.628685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:40:48.628685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:40:48.628685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 5cab8255c6134aebb9c93493a2342cb1 May 28 03:40:48.628685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:40:48.628685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-ServerRescueNegativeTestJSON-968064324-project May 28 03:40:48.628685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:40:48.628685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:40:48.628685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: b6709eed72ca46468fd298929ae0f764 May 28 03:40:48.628685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-ServerRescueNegativeTestJSON-968064324-project May 28 03:40:48.628685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:40:48.628685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Cannot 'detach_volume' instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 while it is in vm_state rescued May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'server_id': '13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690', 'id': '3e4c9aa3-01fe-4aaa-bdfd-4cf2851d934c'} May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 in vm_state rescued. Cannot detach_volume while the instance is in this state. May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:40:48.629603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 while it is in vm_state rescued May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 while it is in vm_state rescued May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:40:48.630858 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c44c2750-73c7-4484-83d5-1e982ba52a85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Returning 409 to user: Cannot 'detach_volume' instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 while it is in vm_state rescued {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:40:48.631824 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c44c2750-73c7-4484-83d5-1e982ba52a85 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "DELETE /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments/3e4c9aa3-01fe-4aaa-bdfd-4cf2851d934c" status: 409 len: 152 microversion: 2.1 time: 0.137105 May 28 03:40:48.631824 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 362/707] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:40:48 2022] DELETE /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments/3e4c9aa3-01fe-4aaa-bdfd-4cf2851d934c => generated 152 bytes in 139 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) May 28 03:40:48.639250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b8058a61-b8ae-48d0-bc14-4781f67ea11b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:48.639913 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b8058a61-b8ae-48d0-bc14-4781f67ea11b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:48.647978 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b8058a61-b8ae-48d0-bc14-4781f67ea11b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.648186 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b8058a61-b8ae-48d0-bc14-4781f67ea11b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:48.648970 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b8058a61-b8ae-48d0-bc14-4781f67ea11b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.649450 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b8058a61-b8ae-48d0-bc14-4781f67ea11b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:48.695388 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:48.718816 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0d947f75-1689-45c6-9a4f-0d5d41770292 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "DELETE /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17/os-volume_attachments/bd590ac9-b81b-4c8e-975d-ee237eda6c4e" status: 202 len: 0 microversion: 2.1 time: 0.402874 May 28 03:40:48.722769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 346/708] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:40:48 2022] DELETE /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17/os-volume_attachments/bd590ac9-b81b-4c8e-975d-ee237eda6c4e => generated 0 bytes in 408 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 28 03:40:48.773940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1e54761e-d42b-4d8f-a54e-2b12a98f745f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:48.774792 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1e54761e-d42b-4d8f-a54e-2b12a98f745f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:48.787050 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1e54761e-d42b-4d8f-a54e-2b12a98f745f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.787489 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1e54761e-d42b-4d8f-a54e-2b12a98f745f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:48.788283 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1e54761e-d42b-4d8f-a54e-2b12a98f745f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.788702 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1e54761e-d42b-4d8f-a54e-2b12a98f745f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:48.869672 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1e54761e-d42b-4d8f-a54e-2b12a98f745f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:48.872148 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b8058a61-b8ae-48d0-bc14-4781f67ea11b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "POST /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/action" status: 202 len: 0 microversion: 2.1 time: 0.236880 May 28 03:40:48.872848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 363/709] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:40:48 2022] POST /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/action => generated 0 bytes in 239 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 28 03:40:48.881953 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4c1862e9-58b1-4094-a311-6f272ee08cd2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:48.882962 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4c1862e9-58b1-4094-a311-6f272ee08cd2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:48.883617 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1e54761e-d42b-4d8f-a54e-2b12a98f745f tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1306 microversion: 2.1 time: 0.113731 May 28 03:40:48.895098 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 347/710] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:48 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1306 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:48.898469 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c1862e9-58b1-4094-a311-6f272ee08cd2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.898469 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c1862e9-58b1-4094-a311-6f272ee08cd2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:48.898469 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c1862e9-58b1-4094-a311-6f272ee08cd2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:48.898469 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4c1862e9-58b1-4094-a311-6f272ee08cd2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:49.035313 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4c1862e9-58b1-4094-a311-6f272ee08cd2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:49.043674 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4c1862e9-58b1-4094-a311-6f272ee08cd2 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1723 microversion: 2.1 time: 0.166502 May 28 03:40:49.044238 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 364/711] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:48 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:49.911664 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4d278ef5-5ccd-458f-8492-7f1e4d10d61e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:49.912756 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4d278ef5-5ccd-458f-8492-7f1e4d10d61e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:49.919711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d278ef5-5ccd-458f-8492-7f1e4d10d61e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:49.920150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d278ef5-5ccd-458f-8492-7f1e4d10d61e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:49.920910 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d278ef5-5ccd-458f-8492-7f1e4d10d61e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:49.921432 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d278ef5-5ccd-458f-8492-7f1e4d10d61e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:50.061656 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-cc9e5caa-64dd-4ee1-b84a-0a7b55c0d7c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:50.062197 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-cc9e5caa-64dd-4ee1-b84a-0a7b55c0d7c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:50.068986 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cc9e5caa-64dd-4ee1-b84a-0a7b55c0d7c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:50.069543 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cc9e5caa-64dd-4ee1-b84a-0a7b55c0d7c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:50.070401 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cc9e5caa-64dd-4ee1-b84a-0a7b55c0d7c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:50.070837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cc9e5caa-64dd-4ee1-b84a-0a7b55c0d7c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:50.211085 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-cc9e5caa-64dd-4ee1-b84a-0a7b55c0d7c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:50.220001 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-cc9e5caa-64dd-4ee1-b84a-0a7b55c0d7c4 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1723 microversion: 2.1 time: 0.163229 May 28 03:40:50.220563 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 365/712] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:50 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:50.244905 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4d278ef5-5ccd-458f-8492-7f1e4d10d61e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:50.252869 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4d278ef5-5ccd-458f-8492-7f1e4d10d61e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.351495 May 28 03:40:50.253470 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 348/713] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:49 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:50.761848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-965b0c0d-08c1-4ee4-a128-77eb2180151d req-f45c2b85-c16d-460f-8cac-be7ed45907ab service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "43942790-5855-4ed1-9ce4-8f02a3092edd", "tag": "741e41e9-4e3d-40e3-80a8-4a50ecab9f1e"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:50.770560 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-965b0c0d-08c1-4ee4-a128-77eb2180151d req-f45c2b85-c16d-460f-8cac-be7ed45907ab service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:50.771022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-965b0c0d-08c1-4ee4-a128-77eb2180151d req-f45c2b85-c16d-460f-8cac-be7ed45907ab service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:50.790055 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-965b0c0d-08c1-4ee4-a128-77eb2180151d req-f45c2b85-c16d-460f-8cac-be7ed45907ab service nova] Creating event network-changed:741e41e9-4e3d-40e3-80a8-4a50ecab9f1e for instance 43942790-5855-4ed1-9ce4-8f02a3092edd on hv-cinder-84336 May 28 03:40:50.795930 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-965b0c0d-08c1-4ee4-a128-77eb2180151d req-f45c2b85-c16d-460f-8cac-be7ed45907ab service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039253 May 28 03:40:50.796579 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 366/714] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:50.902151 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-115fd309-869b-4617-8e60-9e5883b10f89 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:50.902869 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-115fd309-869b-4617-8e60-9e5883b10f89 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:50.909926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-115fd309-869b-4617-8e60-9e5883b10f89 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:50.909926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-115fd309-869b-4617-8e60-9e5883b10f89 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:50.912652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-115fd309-869b-4617-8e60-9e5883b10f89 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:50.913182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-115fd309-869b-4617-8e60-9e5883b10f89 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:50.939993 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-115fd309-869b-4617-8e60-9e5883b10f89 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:40:51.186524 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-115fd309-869b-4617-8e60-9e5883b10f89 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "DELETE /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 204 len: 0 microversion: 2.1 time: 0.287922 May 28 03:40:51.187007 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 349/715] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:40:50 2022] DELETE /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:40:51.195648 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d125cb17-7a99-440a-bba6-4b164e81d9b6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:51.197836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d125cb17-7a99-440a-bba6-4b164e81d9b6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:51.203357 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d125cb17-7a99-440a-bba6-4b164e81d9b6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:51.203805 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d125cb17-7a99-440a-bba6-4b164e81d9b6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:51.204570 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d125cb17-7a99-440a-bba6-4b164e81d9b6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:51.205006 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d125cb17-7a99-440a-bba6-4b164e81d9b6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:51.239880 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4e5e1c5c-71d0-4292-b462-d0f9fbabb2cf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:51.240583 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4e5e1c5c-71d0-4292-b462-d0f9fbabb2cf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:51.249632 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4e5e1c5c-71d0-4292-b462-d0f9fbabb2cf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:51.250578 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4e5e1c5c-71d0-4292-b462-d0f9fbabb2cf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:51.251411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4e5e1c5c-71d0-4292-b462-d0f9fbabb2cf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:51.251847 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4e5e1c5c-71d0-4292-b462-d0f9fbabb2cf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:51.339136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d125cb17-7a99-440a-bba6-4b164e81d9b6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:51.347265 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d125cb17-7a99-440a-bba6-4b164e81d9b6 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1514 microversion: 2.1 time: 0.155865 May 28 03:40:51.347846 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 367/716] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:51 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1514 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:51.352301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0d9869e6-373c-4f4c-afc1-78f21ebe549d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:51.353128 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0d9869e6-373c-4f4c-afc1-78f21ebe549d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:51.359241 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0d9869e6-373c-4f4c-afc1-78f21ebe549d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:51.361394 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0d9869e6-373c-4f4c-afc1-78f21ebe549d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:51.361394 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0d9869e6-373c-4f4c-afc1-78f21ebe549d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:51.361394 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0d9869e6-373c-4f4c-afc1-78f21ebe549d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:51.390630 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4e5e1c5c-71d0-4292-b462-d0f9fbabb2cf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:51.398770 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4e5e1c5c-71d0-4292-b462-d0f9fbabb2cf tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1723 microversion: 2.1 time: 0.165031 May 28 03:40:51.399350 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 350/717] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:51 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1723 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:51.500195 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0d9869e6-373c-4f4c-afc1-78f21ebe549d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:51.507624 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0d9869e6-373c-4f4c-afc1-78f21ebe549d tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.158848 May 28 03:40:51.508191 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 368/718] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:51 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:52.364442 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-310933ef-b249-4b86-b730-e32bf95f625f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:52.364973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-310933ef-b249-4b86-b730-e32bf95f625f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:52.372457 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-310933ef-b249-4b86-b730-e32bf95f625f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:52.372899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-310933ef-b249-4b86-b730-e32bf95f625f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:52.373692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-310933ef-b249-4b86-b730-e32bf95f625f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:52.374228 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-310933ef-b249-4b86-b730-e32bf95f625f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:52.415597 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f69ce57f-ef30-4ac6-bf1f-b545ff6a70da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:52.416231 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f69ce57f-ef30-4ac6-bf1f-b545ff6a70da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:52.423319 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f69ce57f-ef30-4ac6-bf1f-b545ff6a70da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:52.423768 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f69ce57f-ef30-4ac6-bf1f-b545ff6a70da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:52.424532 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f69ce57f-ef30-4ac6-bf1f-b545ff6a70da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:52.425229 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f69ce57f-ef30-4ac6-bf1f-b545ff6a70da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:52.512935 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-310933ef-b249-4b86-b730-e32bf95f625f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:52.521950 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-310933ef-b249-4b86-b730-e32bf95f625f tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1514 microversion: 2.1 time: 0.161340 May 28 03:40:52.522815 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 351/719] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:52 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1514 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:52.529255 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-992a5640-0556-42b9-a395-92dc9d3abf86 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:52.529873 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-992a5640-0556-42b9-a395-92dc9d3abf86 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:52.540683 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-992a5640-0556-42b9-a395-92dc9d3abf86 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:52.541137 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-992a5640-0556-42b9-a395-92dc9d3abf86 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:52.542388 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-992a5640-0556-42b9-a395-92dc9d3abf86 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:52.543400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-992a5640-0556-42b9-a395-92dc9d3abf86 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:52.562636 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f69ce57f-ef30-4ac6-bf1f-b545ff6a70da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:52.570603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f69ce57f-ef30-4ac6-bf1f-b545ff6a70da tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1723 microversion: 2.1 time: 0.158769 May 28 03:40:52.571271 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 369/720] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:52 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:52.672384 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-992a5640-0556-42b9-a395-92dc9d3abf86 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:52.679774 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-992a5640-0556-42b9-a395-92dc9d3abf86 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.155687 May 28 03:40:52.680339 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 352/721] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:52 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:53.538786 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5683e3f3-8fbb-4de9-8b9d-252184dc4d60 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:53.539397 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5683e3f3-8fbb-4de9-8b9d-252184dc4d60 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:53.546444 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5683e3f3-8fbb-4de9-8b9d-252184dc4d60 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:53.546898 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5683e3f3-8fbb-4de9-8b9d-252184dc4d60 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:53.547688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5683e3f3-8fbb-4de9-8b9d-252184dc4d60 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:53.548124 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5683e3f3-8fbb-4de9-8b9d-252184dc4d60 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:53.588362 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4fbce4dd-3d09-4702-b522-2597d8c840e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:53.589070 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4fbce4dd-3d09-4702-b522-2597d8c840e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:53.595523 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4fbce4dd-3d09-4702-b522-2597d8c840e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:53.595966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4fbce4dd-3d09-4702-b522-2597d8c840e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:53.596729 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4fbce4dd-3d09-4702-b522-2597d8c840e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:53.597304 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4fbce4dd-3d09-4702-b522-2597d8c840e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:53.681525 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5683e3f3-8fbb-4de9-8b9d-252184dc4d60 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:53.689629 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5683e3f3-8fbb-4de9-8b9d-252184dc4d60 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1514 microversion: 2.1 time: 0.154639 May 28 03:40:53.690283 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 370/722] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:53 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1514 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:53.696463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d17f8000-bc2a-4e9c-b1b5-5405c6b0de2c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:53.697304 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d17f8000-bc2a-4e9c-b1b5-5405c6b0de2c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:53.703924 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d17f8000-bc2a-4e9c-b1b5-5405c6b0de2c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:53.707166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d17f8000-bc2a-4e9c-b1b5-5405c6b0de2c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:53.707166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d17f8000-bc2a-4e9c-b1b5-5405c6b0de2c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:53.707166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d17f8000-bc2a-4e9c-b1b5-5405c6b0de2c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:53.729160 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4fbce4dd-3d09-4702-b522-2597d8c840e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:53.737823 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4fbce4dd-3d09-4702-b522-2597d8c840e9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1723 microversion: 2.1 time: 0.153331 May 28 03:40:53.737823 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 353/723] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:53 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:53.833317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d17f8000-bc2a-4e9c-b1b5-5405c6b0de2c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:53.841238 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d17f8000-bc2a-4e9c-b1b5-5405c6b0de2c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.148371 May 28 03:40:53.841790 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 371/724] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:53 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:54.706696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3ce1c5a6-6d4f-42e9-a56d-449b8b3766b8 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:54.707417 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3ce1c5a6-6d4f-42e9-a56d-449b8b3766b8 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:54.714120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ce1c5a6-6d4f-42e9-a56d-449b8b3766b8 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:54.714645 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ce1c5a6-6d4f-42e9-a56d-449b8b3766b8 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:54.715995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ce1c5a6-6d4f-42e9-a56d-449b8b3766b8 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:54.716527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3ce1c5a6-6d4f-42e9-a56d-449b8b3766b8 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:54.764503 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c01dbf86-0485-4b3d-95e0-eace7c06d3b9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:54.768680 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c01dbf86-0485-4b3d-95e0-eace7c06d3b9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:54.771725 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c01dbf86-0485-4b3d-95e0-eace7c06d3b9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:54.772172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c01dbf86-0485-4b3d-95e0-eace7c06d3b9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:54.773021 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c01dbf86-0485-4b3d-95e0-eace7c06d3b9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:54.773487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c01dbf86-0485-4b3d-95e0-eace7c06d3b9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:54.907755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c01dbf86-0485-4b3d-95e0-eace7c06d3b9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:54.916587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c01dbf86-0485-4b3d-95e0-eace7c06d3b9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1729 microversion: 2.1 time: 0.164233 May 28 03:40:54.917199 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 372/725] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:54 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1729 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:54.922169 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-613ed9dc-1689-45d8-947d-f2f73af69bef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:54.922780 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-613ed9dc-1689-45d8-947d-f2f73af69bef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:54.929561 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-613ed9dc-1689-45d8-947d-f2f73af69bef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:54.930198 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-613ed9dc-1689-45d8-947d-f2f73af69bef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:54.930987 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-613ed9dc-1689-45d8-947d-f2f73af69bef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:54.931427 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-613ed9dc-1689-45d8-947d-f2f73af69bef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:55.075645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-613ed9dc-1689-45d8-947d-f2f73af69bef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:55.085564 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-613ed9dc-1689-45d8-947d-f2f73af69bef tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.167304 May 28 03:40:55.086133 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 373/726] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:54 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:55.091209 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-67842d66-6206-4aef-9620-7ada92b15600 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:55.092249 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-67842d66-6206-4aef-9620-7ada92b15600 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:55.097817 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-67842d66-6206-4aef-9620-7ada92b15600 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:55.098267 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-67842d66-6206-4aef-9620-7ada92b15600 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:55.099045 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-67842d66-6206-4aef-9620-7ada92b15600 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:55.099481 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-67842d66-6206-4aef-9620-7ada92b15600 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:55.178930 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-67842d66-6206-4aef-9620-7ada92b15600 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.091663 May 28 03:40:55.179630 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 374/727] 10.222.0.41 () {58 vars in 1274 bytes} [Sat May 28 03:40:55 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-interface => generated 285 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:55.192433 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3ce1c5a6-6d4f-42e9-a56d-449b8b3766b8 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:55.200552 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3ce1c5a6-6d4f-42e9-a56d-449b8b3766b8 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1514 microversion: 2.1 time: 0.497666 May 28 03:40:55.208003 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 354/728] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:54 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1514 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:55.705352 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-d6a4e164-9887-4fa1-9c83-e128e60705fb req-22ea4d5d-070d-4762-bcb5-6dde77b31470 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1eaf0b19-b250-4970-adb1-1cdad09fff17", "name": "network-vif-deleted", "tag": "8fe82109-26bb-498f-a1a1-825762536457"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:55.709816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=2 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:55.710735 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=6 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:55.716725 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-d6a4e164-9887-4fa1-9c83-e128e60705fb req-22ea4d5d-070d-4762-bcb5-6dde77b31470 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:55.717298 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-d6a4e164-9887-4fa1-9c83-e128e60705fb req-22ea4d5d-070d-4762-bcb5-6dde77b31470 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:55.730057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-d6a4e164-9887-4fa1-9c83-e128e60705fb req-22ea4d5d-070d-4762-bcb5-6dde77b31470 service nova] Creating event network-vif-deleted:8fe82109-26bb-498f-a1a1-825762536457 for instance 1eaf0b19-b250-4970-adb1-1cdad09fff17 on hv-cinder-84336 May 28 03:40:55.735411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-d6a4e164-9887-4fa1-9c83-e128e60705fb req-22ea4d5d-070d-4762-bcb5-6dde77b31470 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034988 May 28 03:40:55.736004 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 375/729] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:56.102496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1a278eb5-454a-46da-8735-48c141aac9b3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:56.103038 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1a278eb5-454a-46da-8735-48c141aac9b3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:56.105029 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=6 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:56.110806 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1a278eb5-454a-46da-8735-48c141aac9b3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:56.111279 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1a278eb5-454a-46da-8735-48c141aac9b3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:56.112033 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1a278eb5-454a-46da-8735-48c141aac9b3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:56.112465 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1a278eb5-454a-46da-8735-48c141aac9b3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:56.205894 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-c691bef4-4b1c-4231-b78e-b31892a2bafa req-c690a712-a0e4-4104-9a2b-18e3a720ebbe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690", "tag": "ea42d3d0-51a0-41a4-b165-b66f57c57a62"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:56.226977 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-c691bef4-4b1c-4231-b78e-b31892a2bafa req-c690a712-a0e4-4104-9a2b-18e3a720ebbe service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:56.227842 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-c691bef4-4b1c-4231-b78e-b31892a2bafa req-c690a712-a0e4-4104-9a2b-18e3a720ebbe service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:56.243552 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-c691bef4-4b1c-4231-b78e-b31892a2bafa req-c690a712-a0e4-4104-9a2b-18e3a720ebbe service nova] Creating event network-changed:ea42d3d0-51a0-41a4-b165-b66f57c57a62 for instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 on hv-cinder-84336 May 28 03:40:56.248496 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-c691bef4-4b1c-4231-b78e-b31892a2bafa req-c690a712-a0e4-4104-9a2b-18e3a720ebbe service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048458 May 28 03:40:56.249087 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 376/730] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:40:56.257094 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-371be290-c819-42b6-9341-3bd7a748be5e tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:56.257821 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-371be290-c819-42b6-9341-3bd7a748be5e tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:56.258960 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1a278eb5-454a-46da-8735-48c141aac9b3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:56.264585 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-371be290-c819-42b6-9341-3bd7a748be5e tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:56.266831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-371be290-c819-42b6-9341-3bd7a748be5e tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:56.266831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-371be290-c819-42b6-9341-3bd7a748be5e tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:56.266831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-371be290-c819-42b6-9341-3bd7a748be5e tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:56.267127 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1a278eb5-454a-46da-8735-48c141aac9b3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.168151 May 28 03:40:56.270625 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 355/731] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:56 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:56.297811 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-371be290-c819-42b6-9341-3bd7a748be5e tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:56.298264 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-371be290-c819-42b6-9341-3bd7a748be5e tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:56.298719 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-371be290-c819-42b6-9341-3bd7a748be5e tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lazy-loading 'fault' on Instance uuid 1eaf0b19-b250-4970-adb1-1cdad09fff17 {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:40:56.342603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-371be290-c819-42b6-9341-3bd7a748be5e tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:56.350046 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-371be290-c819-42b6-9341-3bd7a748be5e tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1309 microversion: 2.1 time: 0.097009 May 28 03:40:56.350613 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 377/732] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:56 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1309 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:57.284614 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a47a6dbf-986e-43e0-8c97-a0cb73924802 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:57.285829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a47a6dbf-986e-43e0-8c97-a0cb73924802 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:57.292417 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a47a6dbf-986e-43e0-8c97-a0cb73924802 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:57.292920 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a47a6dbf-986e-43e0-8c97-a0cb73924802 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:57.294290 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a47a6dbf-986e-43e0-8c97-a0cb73924802 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:57.294903 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a47a6dbf-986e-43e0-8c97-a0cb73924802 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:57.367744 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-71ede1d2-6aeb-4b08-b8e0-70f353060a63 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:57.368373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-71ede1d2-6aeb-4b08-b8e0-70f353060a63 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:57.375426 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-71ede1d2-6aeb-4b08-b8e0-70f353060a63 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:57.375872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-71ede1d2-6aeb-4b08-b8e0-70f353060a63 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:57.376876 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-71ede1d2-6aeb-4b08-b8e0-70f353060a63 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:57.377519 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-71ede1d2-6aeb-4b08-b8e0-70f353060a63 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:57.412270 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-71ede1d2-6aeb-4b08-b8e0-70f353060a63 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:57.417462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-71ede1d2-6aeb-4b08-b8e0-70f353060a63 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:57.417462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-71ede1d2-6aeb-4b08-b8e0-70f353060a63 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lazy-loading 'fault' on Instance uuid 1eaf0b19-b250-4970-adb1-1cdad09fff17 {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:40:57.471328 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-71ede1d2-6aeb-4b08-b8e0-70f353060a63 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af5fa544bc6440c9830d25e4ad4d5987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e302a2cd63ba46d7a6cef952ddb89242', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:57.481521 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-71ede1d2-6aeb-4b08-b8e0-70f353060a63 tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 200 len: 1309 microversion: 2.1 time: 0.117801 May 28 03:40:57.482317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 378/733] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:57 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 1309 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:40:57.508165 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a47a6dbf-986e-43e0-8c97-a0cb73924802 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:57.516078 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a47a6dbf-986e-43e0-8c97-a0cb73924802 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.235662 May 28 03:40:57.516660 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 356/734] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:57 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:58.263731 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-0387c3b9-1b7e-4f3b-8ada-9a6b3ab07956 req-30b2c125-da56-4d17-81fb-a1ed818ff67f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690", "tag": "ea42d3d0-51a0-41a4-b165-b66f57c57a62"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:40:58.271672 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-0387c3b9-1b7e-4f3b-8ada-9a6b3ab07956 req-30b2c125-da56-4d17-81fb-a1ed818ff67f service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:58.277316 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-0387c3b9-1b7e-4f3b-8ada-9a6b3ab07956 req-30b2c125-da56-4d17-81fb-a1ed818ff67f service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:58.284894 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-0387c3b9-1b7e-4f3b-8ada-9a6b3ab07956 req-30b2c125-da56-4d17-81fb-a1ed818ff67f service nova] Creating event network-changed:ea42d3d0-51a0-41a4-b165-b66f57c57a62 for instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 on hv-cinder-84336 May 28 03:40:58.289998 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-0387c3b9-1b7e-4f3b-8ada-9a6b3ab07956 req-30b2c125-da56-4d17-81fb-a1ed818ff67f service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030665 May 28 03:40:58.290585 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 379/735] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:40:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:40:58.498868 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1dab4b68-54c5-4635-a9fb-2671af992afc tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:58.499618 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1dab4b68-54c5-4635-a9fb-2671af992afc tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] [instance: 1eaf0b19-b250-4970-adb1-1cdad09fff17] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:58.507838 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dab4b68-54c5-4635-a9fb-2671af992afc tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:58.515731 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dab4b68-54c5-4635-a9fb-2671af992afc tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:58.515731 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dab4b68-54c5-4635-a9fb-2671af992afc tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:58.515731 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dab4b68-54c5-4635-a9fb-2671af992afc tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:58.525688 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-1dab4b68-54c5-4635-a9fb-2671af992afc tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] HTTP exception thrown: Instance 1eaf0b19-b250-4970-adb1-1cdad09fff17 could not be found. May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1dab4b68-54c5-4635-a9fb-2671af992afc tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Request method failure captured: May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 HTTP/1.1 May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZknNw6OgVOskXFoQ_7IdZXggvUl-unozmFQ1gLcu8_rV9Si5p65dR_LKNqmNofwjPsnv7CWUdOlv9gXVmJrrTITvBGr_I96CH6HA8h_mLgkldw7kHlMlioYEk5Bg8nj2-roWp6ZC6_tVCMbLwFEWkmTbMWtfbMtd9J510X3jUpxWmw May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: e302a2cd63ba46d7a6cef952ddb89242 May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-2104603578 May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:40:58.527747 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/e302a2cd63ba46d7a6cef952ddb89242"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/e302a2cd63ba46d7a6cef952ddb89242"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/e302a2cd63ba46d7a6cef952ddb89242"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_e302a2cd63ba46d7a6cef952ddb89242"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-VolumesSnapshotTestJSON-2104603578 May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: e302a2cd63ba46d7a6cef952ddb89242 May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-2104603578 May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-VolumesSnapshotTestJSON-2104603578-project May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: af5fa544bc6440c9830d25e4ad4d5987 May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-VolumesSnapshotTestJSON-2104603578-project May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 1eaf0b19-b250-4970-adb1-1cdad09fff17 could not be found. May 28 03:40:58.529464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '1eaf0b19-b250-4970-adb1-1cdad09fff17'} May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1eaf0b19-b250-4970-adb1-1cdad09fff17 could not be found. May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:40:58.530477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1eaf0b19-b250-4970-adb1-1cdad09fff17 could not be found. May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1eaf0b19-b250-4970-adb1-1cdad09fff17 could not be found. May 28 03:40:58.531979 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:40:58.539521 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-744d19fe-4d61-44e0-a7a9-e6f488fce23b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:58.540324 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-744d19fe-4d61-44e0-a7a9-e6f488fce23b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:58.550041 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1dab4b68-54c5-4635-a9fb-2671af992afc tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] Returning 404 to user: Instance 1eaf0b19-b250-4970-adb1-1cdad09fff17 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:40:58.550041 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1dab4b68-54c5-4635-a9fb-2671af992afc tempest-VolumesSnapshotTestJSON-2104603578 tempest-VolumesSnapshotTestJSON-2104603578-project] 10.222.0.41 "GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17" status: 404 len: 111 microversion: 2.1 time: 0.051181 May 28 03:40:58.550041 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 357/736] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:58 2022] GET /compute/v2.1/servers/1eaf0b19-b250-4970-adb1-1cdad09fff17 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:40:58.551938 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-744d19fe-4d61-44e0-a7a9-e6f488fce23b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:58.552321 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-744d19fe-4d61-44e0-a7a9-e6f488fce23b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:58.553155 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-744d19fe-4d61-44e0-a7a9-e6f488fce23b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:58.553569 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-744d19fe-4d61-44e0-a7a9-e6f488fce23b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:58.709881 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-744d19fe-4d61-44e0-a7a9-e6f488fce23b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:58.717626 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-744d19fe-4d61-44e0-a7a9-e6f488fce23b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.182683 May 28 03:40:58.718187 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 380/737] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:58 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:40:59.734911 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2e57691a-ae26-4c38-96a6-de3f39ceca71 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:40:59.735498 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2e57691a-ae26-4c38-96a6-de3f39ceca71 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:40:59.742130 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2e57691a-ae26-4c38-96a6-de3f39ceca71 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:59.742586 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2e57691a-ae26-4c38-96a6-de3f39ceca71 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:59.743333 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2e57691a-ae26-4c38-96a6-de3f39ceca71 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:40:59.743759 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2e57691a-ae26-4c38-96a6-de3f39ceca71 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:40:59.875909 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2e57691a-ae26-4c38-96a6-de3f39ceca71 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:40:59.883425 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2e57691a-ae26-4c38-96a6-de3f39ceca71 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.153015 May 28 03:40:59.883975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 358/738] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:40:59 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:00.409206 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-236c28b9-45b7-450f-9170-d3bf2b0f64d7 req-8119718f-46c3-49f0-8c8d-344652623036 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43942790-5855-4ed1-9ce4-8f02a3092edd", "name": "network-vif-plugged", "status": "completed", "tag": "741e41e9-4e3d-40e3-80a8-4a50ecab9f1e"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:00.417478 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-236c28b9-45b7-450f-9170-d3bf2b0f64d7 req-8119718f-46c3-49f0-8c8d-344652623036 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:00.418058 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-236c28b9-45b7-450f-9170-d3bf2b0f64d7 req-8119718f-46c3-49f0-8c8d-344652623036 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:00.428846 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-236c28b9-45b7-450f-9170-d3bf2b0f64d7 req-8119718f-46c3-49f0-8c8d-344652623036 service nova] Creating event network-vif-plugged:741e41e9-4e3d-40e3-80a8-4a50ecab9f1e for instance 43942790-5855-4ed1-9ce4-8f02a3092edd on hv-cinder-84336 May 28 03:41:00.433820 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-236c28b9-45b7-450f-9170-d3bf2b0f64d7 req-8119718f-46c3-49f0-8c8d-344652623036 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028601 May 28 03:41:00.434411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 381/739] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:41:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:00.899910 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-58d9b5b7-984b-421f-a135-5437d21e402e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:00.900521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-58d9b5b7-984b-421f-a135-5437d21e402e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:00.907252 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-58d9b5b7-984b-421f-a135-5437d21e402e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:00.907956 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-58d9b5b7-984b-421f-a135-5437d21e402e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:00.909319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-58d9b5b7-984b-421f-a135-5437d21e402e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:00.909773 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-58d9b5b7-984b-421f-a135-5437d21e402e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:01.050923 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-58d9b5b7-984b-421f-a135-5437d21e402e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:01.054411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:01.058705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-58d9b5b7-984b-421f-a135-5437d21e402e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.162700 May 28 03:41:01.059263 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 359/740] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:00 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:02.076362 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5af35a4b-719e-40b8-9a62-e6ca6ff186e9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:02.077057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5af35a4b-719e-40b8-9a62-e6ca6ff186e9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:02.083232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5af35a4b-719e-40b8-9a62-e6ca6ff186e9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:02.083613 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5af35a4b-719e-40b8-9a62-e6ca6ff186e9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:02.088198 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5af35a4b-719e-40b8-9a62-e6ca6ff186e9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:02.088198 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5af35a4b-719e-40b8-9a62-e6ca6ff186e9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:02.211430 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5af35a4b-719e-40b8-9a62-e6ca6ff186e9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:02.214919 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:UPDATE=1,nova_cell1:SELECT=98,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:02.219764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5af35a4b-719e-40b8-9a62-e6ca6ff186e9 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.148104 May 28 03:41:02.220325 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 382/741] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:02 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:03.291354 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-dc2b5ae9-d953-4c91-8263-2da933d37047 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:03.291997 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-dc2b5ae9-d953-4c91-8263-2da933d37047 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:03.300169 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc2b5ae9-d953-4c91-8263-2da933d37047 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:03.302102 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc2b5ae9-d953-4c91-8263-2da933d37047 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:03.302102 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc2b5ae9-d953-4c91-8263-2da933d37047 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:03.302102 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dc2b5ae9-d953-4c91-8263-2da933d37047 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:03.428736 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-dc2b5ae9-d953-4c91-8263-2da933d37047 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:03.436652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-dc2b5ae9-d953-4c91-8263-2da933d37047 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.203708 May 28 03:41:03.437244 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 360/742] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:03 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:04.454481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5700f310-5a21-4d70-b718-f57ade98eec5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:04.455101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5700f310-5a21-4d70-b718-f57ade98eec5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:04.460948 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e239f1bb-a46c-48d2-8e80-10f54ebf6cf7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:04.461487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e239f1bb-a46c-48d2-8e80-10f54ebf6cf7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:04.461962 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5700f310-5a21-4d70-b718-f57ade98eec5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:04.462404 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5700f310-5a21-4d70-b718-f57ade98eec5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:04.463163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5700f310-5a21-4d70-b718-f57ade98eec5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:04.463594 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5700f310-5a21-4d70-b718-f57ade98eec5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:04.468906 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e239f1bb-a46c-48d2-8e80-10f54ebf6cf7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:04.469386 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e239f1bb-a46c-48d2-8e80-10f54ebf6cf7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:04.470160 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e239f1bb-a46c-48d2-8e80-10f54ebf6cf7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:04.470602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e239f1bb-a46c-48d2-8e80-10f54ebf6cf7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:04.494243 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e239f1bb-a46c-48d2-8e80-10f54ebf6cf7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/ips" status: 200 len: 149 microversion: 2.1 time: 0.141061 May 28 03:41:04.494835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 383/743] 10.222.0.41 () {58 vars in 1247 bytes} [Sat May 28 03:41:04 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/ips => generated 149 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:04.598620 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5700f310-5a21-4d70-b718-f57ade98eec5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:04.605866 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5700f310-5a21-4d70-b718-f57ade98eec5 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1361 microversion: 2.1 time: 0.156021 May 28 03:41:04.606964 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 361/744] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:04 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1361 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:41:05.624279 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1bb6f6ae-9dd3-441b-8890-a65ce62b0a46 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:05.625271 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1bb6f6ae-9dd3-441b-8890-a65ce62b0a46 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:05.633021 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1bb6f6ae-9dd3-441b-8890-a65ce62b0a46 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:05.633501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1bb6f6ae-9dd3-441b-8890-a65ce62b0a46 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:05.634419 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1bb6f6ae-9dd3-441b-8890-a65ce62b0a46 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:05.634863 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1bb6f6ae-9dd3-441b-8890-a65ce62b0a46 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:05.980586 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1bb6f6ae-9dd3-441b-8890-a65ce62b0a46 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:05.988247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1bb6f6ae-9dd3-441b-8890-a65ce62b0a46 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1547 microversion: 2.1 time: 0.368701 May 28 03:41:05.988821 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 384/745] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:05 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1547 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:06.004853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0e551560-5c87-4fbc-90b3-9a26cbe30991 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:06.005502 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0e551560-5c87-4fbc-90b3-9a26cbe30991 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:06.011863 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0e551560-5c87-4fbc-90b3-9a26cbe30991 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:06.012320 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0e551560-5c87-4fbc-90b3-9a26cbe30991 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:06.013705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0e551560-5c87-4fbc-90b3-9a26cbe30991 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:06.014246 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0e551560-5c87-4fbc-90b3-9a26cbe30991 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:06.220803 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b6ce96ae-beb0-4722-bd63-eafd1c543ee6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d5620542-392e-4fd0-b68d-35a39f07929e"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:06.221962 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b6ce96ae-beb0-4722-bd63-eafd1c543ee6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:06.228114 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b6ce96ae-beb0-4722-bd63-eafd1c543ee6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:06.228555 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b6ce96ae-beb0-4722-bd63-eafd1c543ee6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:06.229360 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b6ce96ae-beb0-4722-bd63-eafd1c543ee6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:06.229804 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b6ce96ae-beb0-4722-bd63-eafd1c543ee6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:06.259981 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0e551560-5c87-4fbc-90b3-9a26cbe30991 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.258732 May 28 03:41:06.261496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 362/746] 10.222.0.41 () {58 vars in 1274 bytes} [Sat May 28 03:41:05 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-interface => generated 284 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:07.100318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b6ce96ae-beb0-4722-bd63-eafd1c543ee6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.883325 May 28 03:41:07.100978 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 385/747] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:41:06 2022] POST /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments => generated 194 bytes in 885 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:07.396853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-35b41b5b-907e-4bb5-93bf-e7393b4a021c req-42df4a1a-6a1b-46dc-bb4d-93151a5ec2d8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "43942790-5855-4ed1-9ce4-8f02a3092edd", "tag": "741e41e9-4e3d-40e3-80a8-4a50ecab9f1e"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:07.405781 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-35b41b5b-907e-4bb5-93bf-e7393b4a021c req-42df4a1a-6a1b-46dc-bb4d-93151a5ec2d8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:07.406228 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-35b41b5b-907e-4bb5-93bf-e7393b4a021c req-42df4a1a-6a1b-46dc-bb4d-93151a5ec2d8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:07.419965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-35b41b5b-907e-4bb5-93bf-e7393b4a021c req-42df4a1a-6a1b-46dc-bb4d-93151a5ec2d8 service nova] Creating event network-changed:741e41e9-4e3d-40e3-80a8-4a50ecab9f1e for instance 43942790-5855-4ed1-9ce4-8f02a3092edd on hv-cinder-84336 May 28 03:41:07.425428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-35b41b5b-907e-4bb5-93bf-e7393b4a021c req-42df4a1a-6a1b-46dc-bb4d-93151a5ec2d8 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034179 May 28 03:41:07.426305 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 363/748] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:41:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:17.858437 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5db46e65-d809-4a0d-8afe-e51edefef0e7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d5620542-392e-4fd0-b68d-35a39f07929e"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:17.858437 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5db46e65-d809-4a0d-8afe-e51edefef0e7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:17.858437 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=52 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:17.859349 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=21,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:17.863772 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5db46e65-d809-4a0d-8afe-e51edefef0e7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:17.864219 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5db46e65-d809-4a0d-8afe-e51edefef0e7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:17.864986 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5db46e65-d809-4a0d-8afe-e51edefef0e7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:17.865475 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5db46e65-d809-4a0d-8afe-e51edefef0e7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:17.950196 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-5db46e65-d809-4a0d-8afe-e51edefef0e7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Invalid volume: volume d5620542-392e-4fd0-b68d-35a39f07929e is already attached to instances: c2d3a613-c1f1-49d5-a707-784bdc430953 May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5db46e65-d809-4a0d-8afe-e51edefef0e7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: POST /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments HTTP/1.1 May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 74 May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:41:17.951921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {"volumeAttachment": {"volumeId": "d5620542-392e-4fd0-b68d-35a39f07929e"}} May 28 03:41:17.953529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Invalid volume: volume d5620542-392e-4fd0-b68d-35a39f07929e is already attached to instances: c2d3a613-c1f1-49d5-a707-784bdc430953 May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'server_id': 'c2d3a613-c1f1-49d5-a707-784bdc430953', 'body': {'volumeAttachment': {'volumeId': 'd5620542-392e-4fd0-b68d-35a39f07929e'}}} May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume d5620542-392e-4fd0-b68d-35a39f07929e is already attached to instances: c2d3a613-c1f1-49d5-a707-784bdc430953 May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:17.954673 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume d5620542-392e-4fd0-b68d-35a39f07929e is already attached to instances: c2d3a613-c1f1-49d5-a707-784bdc430953 May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:17.955836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:41:17.957709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:41:17.957709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:41:17.957709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:41:17.957709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume d5620542-392e-4fd0-b68d-35a39f07929e is already attached to instances: c2d3a613-c1f1-49d5-a707-784bdc430953 May 28 03:41:17.957709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:17.957709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5db46e65-d809-4a0d-8afe-e51edefef0e7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 400 to user: Invalid volume: volume d5620542-392e-4fd0-b68d-35a39f07929e is already attached to instances: c2d3a613-c1f1-49d5-a707-784bdc430953 {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:41:17.957709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5db46e65-d809-4a0d-8afe-e51edefef0e7 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.103590 May 28 03:41:17.957709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 386/749] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:41:17 2022] POST /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments => generated 174 bytes in 105 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 28 03:41:18.011821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=48,nova_api:UPDATE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:18.012584 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=18 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:18.016764 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9438d221-0e83-4e3e-aeef-53b57b414b5b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:18.017374 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9438d221-0e83-4e3e-aeef-53b57b414b5b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:18.023615 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9438d221-0e83-4e3e-aeef-53b57b414b5b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:18.024069 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9438d221-0e83-4e3e-aeef-53b57b414b5b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:18.024836 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9438d221-0e83-4e3e-aeef-53b57b414b5b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:18.025340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9438d221-0e83-4e3e-aeef-53b57b414b5b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:18.106790 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-9438d221-0e83-4e3e-aeef-53b57b414b5b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lazy-loading 'services' on Instance uuid c2d3a613-c1f1-49d5-a707-784bdc430953 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:41:18.358048 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9438d221-0e83-4e3e-aeef-53b57b414b5b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments/d5620542-392e-4fd0-b68d-35a39f07929e" status: 202 len: 0 microversion: 2.1 time: 0.346916 May 28 03:41:18.367931 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 364/750] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:41:18 2022] DELETE /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments/d5620542-392e-4fd0-b68d-35a39f07929e => generated 0 bytes in 349 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:41:18.374776 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1c0942f9-c1bd-4de3-96d7-156f7f2af13d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:18.375701 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1c0942f9-c1bd-4de3-96d7-156f7f2af13d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:18.382402 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1c0942f9-c1bd-4de3-96d7-156f7f2af13d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:18.382853 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1c0942f9-c1bd-4de3-96d7-156f7f2af13d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:18.383620 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1c0942f9-c1bd-4de3-96d7-156f7f2af13d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:18.384060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1c0942f9-c1bd-4de3-96d7-156f7f2af13d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:18.413350 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1c0942f9-c1bd-4de3-96d7-156f7f2af13d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042425 May 28 03:41:18.413897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 387/751] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:41:18 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:19.425385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c6be425e-7829-430b-a5cc-c0b75f76b9e3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:19.426366 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c6be425e-7829-430b-a5cc-c0b75f76b9e3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:19.432556 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c6be425e-7829-430b-a5cc-c0b75f76b9e3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:19.432999 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c6be425e-7829-430b-a5cc-c0b75f76b9e3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:19.433824 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c6be425e-7829-430b-a5cc-c0b75f76b9e3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:19.434196 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c6be425e-7829-430b-a5cc-c0b75f76b9e3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:19.462903 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c6be425e-7829-430b-a5cc-c0b75f76b9e3 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042326 May 28 03:41:19.463492 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 365/752] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:41:19 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:20.474852 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-15072799-ef19-411c-9b80-a65f251fd275 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:20.475765 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-15072799-ef19-411c-9b80-a65f251fd275 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:20.482478 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-15072799-ef19-411c-9b80-a65f251fd275 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:20.482917 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-15072799-ef19-411c-9b80-a65f251fd275 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:20.483670 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-15072799-ef19-411c-9b80-a65f251fd275 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:20.484097 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-15072799-ef19-411c-9b80-a65f251fd275 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:20.517479 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-15072799-ef19-411c-9b80-a65f251fd275 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046783 May 28 03:41:20.518040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 388/753] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:41:20 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 28 03:41:20.595691 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a6934f28-0dd1-4143-8b87-008b43c48cd2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:20.596200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a6934f28-0dd1-4143-8b87-008b43c48cd2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:20.602602 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a6934f28-0dd1-4143-8b87-008b43c48cd2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:20.603115 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a6934f28-0dd1-4143-8b87-008b43c48cd2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:20.603778 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a6934f28-0dd1-4143-8b87-008b43c48cd2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:20.604208 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a6934f28-0dd1-4143-8b87-008b43c48cd2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:20.629387 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a6934f28-0dd1-4143-8b87-008b43c48cd2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:41:20.922913 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a6934f28-0dd1-4143-8b87-008b43c48cd2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 204 len: 0 microversion: 2.1 time: 0.330530 May 28 03:41:20.923481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 366/754] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:41:20 2022] DELETE /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:41:20.932173 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-7bd9e90f-af25-4088-b0ee-f0246a4b29d9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:20.932789 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-7bd9e90f-af25-4088-b0ee-f0246a4b29d9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:20.939902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7bd9e90f-af25-4088-b0ee-f0246a4b29d9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:20.940350 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7bd9e90f-af25-4088-b0ee-f0246a4b29d9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:20.941205 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7bd9e90f-af25-4088-b0ee-f0246a4b29d9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:20.941643 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7bd9e90f-af25-4088-b0ee-f0246a4b29d9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:21.069808 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-7bd9e90f-af25-4088-b0ee-f0246a4b29d9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:21.077788 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-7bd9e90f-af25-4088-b0ee-f0246a4b29d9 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1681 microversion: 2.1 time: 0.149854 May 28 03:41:21.078396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 389/755] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:20 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1681 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:22.096443 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-09923ce9-44d0-49d6-b64e-7a828bb50454 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:22.097079 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-09923ce9-44d0-49d6-b64e-7a828bb50454 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:22.108059 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-09923ce9-44d0-49d6-b64e-7a828bb50454 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:22.109091 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-09923ce9-44d0-49d6-b64e-7a828bb50454 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:22.109893 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-09923ce9-44d0-49d6-b64e-7a828bb50454 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:22.110330 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-09923ce9-44d0-49d6-b64e-7a828bb50454 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:22.243502 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-09923ce9-44d0-49d6-b64e-7a828bb50454 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:22.250914 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-09923ce9-44d0-49d6-b64e-7a828bb50454 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1681 microversion: 2.1 time: 0.159424 May 28 03:41:22.252411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 367/756] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:22 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1681 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:23.268425 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3ca499a4-5f79-44c4-b534-cd95a8d8784b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:23.269042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3ca499a4-5f79-44c4-b534-cd95a8d8784b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:23.275503 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ca499a4-5f79-44c4-b534-cd95a8d8784b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:23.275950 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ca499a4-5f79-44c4-b534-cd95a8d8784b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:23.276713 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ca499a4-5f79-44c4-b534-cd95a8d8784b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:23.277169 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3ca499a4-5f79-44c4-b534-cd95a8d8784b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:23.407995 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3ca499a4-5f79-44c4-b534-cd95a8d8784b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:23.415531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3ca499a4-5f79-44c4-b534-cd95a8d8784b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1681 microversion: 2.1 time: 0.151076 May 28 03:41:23.416100 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 390/757] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:23 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1681 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:24.432705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-dbac030c-4a62-433c-9c42-fdaa254d36bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:24.433355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-dbac030c-4a62-433c-9c42-fdaa254d36bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:24.440474 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dbac030c-4a62-433c-9c42-fdaa254d36bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:24.440958 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dbac030c-4a62-433c-9c42-fdaa254d36bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:24.441789 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dbac030c-4a62-433c-9c42-fdaa254d36bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:24.442220 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dbac030c-4a62-433c-9c42-fdaa254d36bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:24.654246 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-dbac030c-4a62-433c-9c42-fdaa254d36bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:24.662509 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-dbac030c-4a62-433c-9c42-fdaa254d36bf tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1681 microversion: 2.1 time: 0.233793 May 28 03:41:24.663078 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 368/758] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:24 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1681 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:25.579135 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-67a65b4c-8ae5-4fd6-bd40-c09d02476bb4 req-f5f36001-e467-45db-91b5-079240cd0603 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c2d3a613-c1f1-49d5-a707-784bdc430953", "name": "network-vif-deleted", "tag": "f1321a9e-949d-4d52-abcc-e48ee7436a3a"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:25.600725 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-67a65b4c-8ae5-4fd6-bd40-c09d02476bb4 req-f5f36001-e467-45db-91b5-079240cd0603 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:25.601320 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-67a65b4c-8ae5-4fd6-bd40-c09d02476bb4 req-f5f36001-e467-45db-91b5-079240cd0603 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:25.616134 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-67a65b4c-8ae5-4fd6-bd40-c09d02476bb4 req-f5f36001-e467-45db-91b5-079240cd0603 service nova] Creating event network-vif-deleted:f1321a9e-949d-4d52-abcc-e48ee7436a3a for instance c2d3a613-c1f1-49d5-a707-784bdc430953 on hv-cinder-84336 May 28 03:41:25.623466 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-67a65b4c-8ae5-4fd6-bd40-c09d02476bb4 req-f5f36001-e467-45db-91b5-079240cd0603 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.205191 May 28 03:41:25.628226 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 391/759] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:41:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:25.679995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f846a0f6-edbc-4ae7-94ff-86aefef3c3de tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:25.681510 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f846a0f6-edbc-4ae7-94ff-86aefef3c3de tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:25.690942 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f846a0f6-edbc-4ae7-94ff-86aefef3c3de tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:25.691659 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f846a0f6-edbc-4ae7-94ff-86aefef3c3de tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:25.692632 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f846a0f6-edbc-4ae7-94ff-86aefef3c3de tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:25.693278 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f846a0f6-edbc-4ae7-94ff-86aefef3c3de tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:25.758924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f846a0f6-edbc-4ae7-94ff-86aefef3c3de tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:25.767887 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f846a0f6-edbc-4ae7-94ff-86aefef3c3de tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 200 len: 1326 microversion: 2.1 time: 0.091820 May 28 03:41:25.768480 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 369/760] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:25 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 1326 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:26.785594 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e54c3f5b-5531-4e1f-999c-179a6ef4f900 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:26.786289 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e54c3f5b-5531-4e1f-999c-179a6ef4f900 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:26.793232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e54c3f5b-5531-4e1f-999c-179a6ef4f900 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:26.794075 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e54c3f5b-5531-4e1f-999c-179a6ef4f900 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:26.794948 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e54c3f5b-5531-4e1f-999c-179a6ef4f900 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:26.795423 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e54c3f5b-5531-4e1f-999c-179a6ef4f900 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:26.809456 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-e54c3f5b-5531-4e1f-999c-179a6ef4f900 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e54c3f5b-5531-4e1f-999c-179a6ef4f900 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 HTTP/1.1 May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:41:26.811068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:41:26.813089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': 'c2d3a613-c1f1-49d5-a707-784bdc430953'} May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:41:26.813910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:41:26.815172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:26.816257 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e54c3f5b-5531-4e1f-999c-179a6ef4f900 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:41:26.816257 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e54c3f5b-5531-4e1f-999c-179a6ef4f900 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 404 len: 111 microversion: 2.1 time: 0.031792 May 28 03:41:26.816257 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 392/761] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:26 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:41:26.821718 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-87ab0c72-44a5-415e-a9fa-0134e775e985 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '.version_select at 0x7feb1fc66af0>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:26.936317 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-87ab0c72-44a5-415e-a9fa-0134e775e985 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-978379138" status: 202 len: 0 microversion: 2.1 time: 0.118285 May 28 03:41:26.937499 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 370/762] 10.222.0.41 () {58 vars in 1217 bytes} [Sat May 28 03:41:26 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-978379138 => generated 0 bytes in 120 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:41:27.772926 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ca1c0d94-5b8c-46a6-8dc9-6b69b4f06607 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: .version_select at 0x7feb1f40e4c0>, body: {"keypair": {"name": "tempest-keypair-1392860992", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLw7T6gwjtqb82i626kkWPWK0f3ITZ2oSaeGLqw3MXGpfvBt/tZzi11c+nFWX6tquE19oVORMms252rhUJjadtY+PyAdiRCRp3xLOLPu0SIDjqyHxmjk0VuXkg8i8bKeSQ=="}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:27.787203 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-ca1c0d94-5b8c-46a6-8dc9-6b69b4f06607 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'key_pairs'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:41:27.791352 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-ca1c0d94-5b8c-46a6-8dc9-6b69b4f06607 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for user 9d420e4b554e4deab21b89094e0547ca and project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'key_pairs'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:41:27.884478 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ca1c0d94-5b8c-46a6-8dc9-6b69b4f06607 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.115406 May 28 03:41:27.885060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 393/763] 10.222.0.41 () {60 vars in 1158 bytes} [Sat May 28 03:41:27 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:30.461745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1100546609", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ee7025a6-8632-48bc-9a4d-e0b6d8e98a79"}], "security_groups": [{"name": "tempest-securitygroup--803183140"}], "key_name": "tempest-keypair-1392860992", "user_data": "}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:30.567298 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.network.neutron [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] validate_networks() for [('ee7025a6-8632-48bc-9a4d-e0b6d8e98a79', None, None, None, None, None)] {{(pid=124605) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:41:30.991765 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Flavor limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:41:30.992192 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Image limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:41:30.992965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Flavor pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:41:30.993495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Image pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:41:30.993909 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:41:31.008153 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:31.008153 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:31.008153 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:31.008153 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:31.023151 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:41:31.027177 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Getting quotas for user 9d420e4b554e4deab21b89094e0547ca and project 68e1a9e256f64e46919b1e362b4f68d8. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:41:31.038322 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Going to run 1 instances... {{(pid=124605) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:41:31.040629 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:41:31.166015 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-dbb11ac6-0ef0-4534-8630-957f4a48187d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.709384 May 28 03:41:31.166859 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 371/764] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:41:30 2022] POST /compute/v2.1/servers => generated 401 bytes in 711 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:41:31.177619 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5477842b-bcbe-4580-8f07-13f627304c2d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:31.178769 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5477842b-bcbe-4580-8f07-13f627304c2d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:31.259383 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5477842b-bcbe-4580-8f07-13f627304c2d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:31.270816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5477842b-bcbe-4580-8f07-13f627304c2d tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1242 microversion: 2.1 time: 0.097167 May 28 03:41:31.271747 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 394/765] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:31 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1242 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:32.289413 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f1a54291-1c4f-44ff-8b9a-27724c488095 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:32.290003 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f1a54291-1c4f-44ff-8b9a-27724c488095 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:32.343571 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f1a54291-1c4f-44ff-8b9a-27724c488095 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:32.350154 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f1a54291-1c4f-44ff-8b9a-27724c488095 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1242 microversion: 2.1 time: 0.065335 May 28 03:41:32.350786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 372/766] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:32 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1242 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:33.368293 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5a27d40e-5929-4c1a-9ae6-1b86a3887307 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:33.368911 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5a27d40e-5929-4c1a-9ae6-1b86a3887307 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:33.375591 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5a27d40e-5929-4c1a-9ae6-1b86a3887307 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:33.376043 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5a27d40e-5929-4c1a-9ae6-1b86a3887307 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:33.376816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5a27d40e-5929-4c1a-9ae6-1b86a3887307 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:33.377297 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5a27d40e-5929-4c1a-9ae6-1b86a3887307 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:33.443792 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5a27d40e-5929-4c1a-9ae6-1b86a3887307 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:33.451477 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5a27d40e-5929-4c1a-9ae6-1b86a3887307 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1294 microversion: 2.1 time: 0.087566 May 28 03:41:33.452599 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 395/767] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:33 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1294 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:34.469523 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6a04cade-ea1f-41d0-a95d-03d42a057562 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:34.470157 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6a04cade-ea1f-41d0-a95d-03d42a057562 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:34.476581 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6a04cade-ea1f-41d0-a95d-03d42a057562 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:34.477018 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6a04cade-ea1f-41d0-a95d-03d42a057562 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:34.479510 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6a04cade-ea1f-41d0-a95d-03d42a057562 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:34.479510 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6a04cade-ea1f-41d0-a95d-03d42a057562 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:34.971740 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6a04cade-ea1f-41d0-a95d-03d42a057562 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:34.979080 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6a04cade-ea1f-41d0-a95d-03d42a057562 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1367 microversion: 2.1 time: 0.513427 May 28 03:41:34.979643 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 373/768] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:34 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1367 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:35.879899 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-bc20e7be-64ca-4f59-924c-4906402a6a5c req-8b167de7-6fb4-44e4-a266-cbb688651364 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f65cf50d-baf3-4a38-9dec-14ff91d5f6ac", "tag": "b60c0498-d47b-4439-b615-1ff9bb9a032a"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:35.895376 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-bc20e7be-64ca-4f59-924c-4906402a6a5c req-8b167de7-6fb4-44e4-a266-cbb688651364 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:35.896010 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-bc20e7be-64ca-4f59-924c-4906402a6a5c req-8b167de7-6fb4-44e4-a266-cbb688651364 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:35.923963 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-bc20e7be-64ca-4f59-924c-4906402a6a5c req-8b167de7-6fb4-44e4-a266-cbb688651364 service nova] Creating event network-changed:b60c0498-d47b-4439-b615-1ff9bb9a032a for instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac on hv-cinder-84336 May 28 03:41:35.929291 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-bc20e7be-64ca-4f59-924c-4906402a6a5c req-8b167de7-6fb4-44e4-a266-cbb688651364 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054666 May 28 03:41:35.929902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 396/769] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:41:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:35.996237 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-07221019-f10c-4893-a3e3-12cbb698e6d6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:35.996842 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-07221019-f10c-4893-a3e3-12cbb698e6d6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:36.009606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-07221019-f10c-4893-a3e3-12cbb698e6d6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:36.010058 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-07221019-f10c-4893-a3e3-12cbb698e6d6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:36.010838 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-07221019-f10c-4893-a3e3-12cbb698e6d6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:36.011277 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-07221019-f10c-4893-a3e3-12cbb698e6d6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:36.268132 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-07221019-f10c-4893-a3e3-12cbb698e6d6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:36.272293 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-07221019-f10c-4893-a3e3-12cbb698e6d6 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1367 microversion: 2.1 time: 0.280136 May 28 03:41:36.272871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 374/770] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:35 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1367 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:37.290945 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-49e4dd07-7e63-4ca9-8a1d-37d7ecb66994 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:37.291771 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-49e4dd07-7e63-4ca9-8a1d-37d7ecb66994 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:37.296332 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a3f91cd1-da01-423d-9764-4a8c00f3c16b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:37.296831 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a3f91cd1-da01-423d-9764-4a8c00f3c16b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:37.298838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49e4dd07-7e63-4ca9-8a1d-37d7ecb66994 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:37.299294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49e4dd07-7e63-4ca9-8a1d-37d7ecb66994 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:37.300067 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49e4dd07-7e63-4ca9-8a1d-37d7ecb66994 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:37.300497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49e4dd07-7e63-4ca9-8a1d-37d7ecb66994 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:37.303690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3f91cd1-da01-423d-9764-4a8c00f3c16b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:37.304167 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3f91cd1-da01-423d-9764-4a8c00f3c16b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:37.305313 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3f91cd1-da01-423d-9764-4a8c00f3c16b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:37.305757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a3f91cd1-da01-423d-9764-4a8c00f3c16b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:37.406028 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-a3f91cd1-da01-423d-9764-4a8c00f3c16b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lazy-loading 'services' on Instance uuid 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:41:37.568463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-49e4dd07-7e63-4ca9-8a1d-37d7ecb66994 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:37.576419 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-49e4dd07-7e63-4ca9-8a1d-37d7ecb66994 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1367 microversion: 2.1 time: 0.290371 May 28 03:41:37.577348 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 397/771] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:37 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1367 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:41:37.687990 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a3f91cd1-da01-423d-9764-4a8c00f3c16b tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "DELETE /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments/3e4c9aa3-01fe-4aaa-bdfd-4cf2851d934c" status: 202 len: 0 microversion: 2.1 time: 0.395801 May 28 03:41:37.688879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 375/772] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:41:37 2022] DELETE /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments/3e4c9aa3-01fe-4aaa-bdfd-4cf2851d934c => generated 0 bytes in 398 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 28 03:41:37.697435 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5f645df9-0ca4-4917-ad41-3737231814ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:37.702008 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5f645df9-0ca4-4917-ad41-3737231814ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:37.706028 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5f645df9-0ca4-4917-ad41-3737231814ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:37.707035 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5f645df9-0ca4-4917-ad41-3737231814ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:37.708308 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5f645df9-0ca4-4917-ad41-3737231814ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:37.709041 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5f645df9-0ca4-4917-ad41-3737231814ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:37.746226 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5f645df9-0ca4-4917-ad41-3737231814ad tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052732 May 28 03:41:37.747287 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 398/773] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:41:37 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:38.596598 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-652c7a3b-c203-4e8d-a604-8b886c0ecbd8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:38.597438 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-652c7a3b-c203-4e8d-a604-8b886c0ecbd8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:38.604820 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-652c7a3b-c203-4e8d-a604-8b886c0ecbd8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:38.605395 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-652c7a3b-c203-4e8d-a604-8b886c0ecbd8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:38.606299 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-652c7a3b-c203-4e8d-a604-8b886c0ecbd8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:38.606926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-652c7a3b-c203-4e8d-a604-8b886c0ecbd8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:38.759855 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b29791e3-1511-478a-8b62-da4c443a70c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:38.762910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b29791e3-1511-478a-8b62-da4c443a70c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:38.778244 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b29791e3-1511-478a-8b62-da4c443a70c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:38.778907 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b29791e3-1511-478a-8b62-da4c443a70c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:38.795000 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b29791e3-1511-478a-8b62-da4c443a70c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:38.800992 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b29791e3-1511-478a-8b62-da4c443a70c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:38.815123 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-652c7a3b-c203-4e8d-a604-8b886c0ecbd8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:38.825172 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-652c7a3b-c203-4e8d-a604-8b886c0ecbd8 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1367 microversion: 2.1 time: 0.232801 May 28 03:41:38.826814 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 376/774] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:38 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1367 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:38.863760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b29791e3-1511-478a-8b62-da4c443a70c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.107992 May 28 03:41:38.864635 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 399/775] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:41:38 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments => generated 197 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:41:39.843800 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c08d8e22-1547-44fc-952c-160d175946a5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:39.844413 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c08d8e22-1547-44fc-952c-160d175946a5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:39.851793 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c08d8e22-1547-44fc-952c-160d175946a5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:39.856210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c08d8e22-1547-44fc-952c-160d175946a5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:39.856210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c08d8e22-1547-44fc-952c-160d175946a5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:39.856210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c08d8e22-1547-44fc-952c-160d175946a5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:39.876150 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8ca43dcd-d840-4d34-be36-ab997b1c6ac8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:39.877949 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8ca43dcd-d840-4d34-be36-ab997b1c6ac8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:39.885890 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8ca43dcd-d840-4d34-be36-ab997b1c6ac8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:39.886598 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8ca43dcd-d840-4d34-be36-ab997b1c6ac8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:39.887551 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8ca43dcd-d840-4d34-be36-ab997b1c6ac8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:39.888149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8ca43dcd-d840-4d34-be36-ab997b1c6ac8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:39.954581 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8ca43dcd-d840-4d34-be36-ab997b1c6ac8 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.082369 May 28 03:41:39.961686 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 400/776] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:41:39 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690/os-volume_attachments => generated 25 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 28 03:41:40.096203 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2e7c3e96-24c6-4feb-b158-c4f782597712 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:40.096933 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2e7c3e96-24c6-4feb-b158-c4f782597712 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:40.103709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2e7c3e96-24c6-4feb-b158-c4f782597712 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:40.104504 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2e7c3e96-24c6-4feb-b158-c4f782597712 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:40.105610 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2e7c3e96-24c6-4feb-b158-c4f782597712 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:40.112106 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2e7c3e96-24c6-4feb-b158-c4f782597712 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:40.122243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c08d8e22-1547-44fc-952c-160d175946a5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:40.134996 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c08d8e22-1547-44fc-952c-160d175946a5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1367 microversion: 2.1 time: 0.293721 May 28 03:41:40.135460 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 377/777] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:39 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1367 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:40.156801 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2e7c3e96-24c6-4feb-b158-c4f782597712 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Going to try to terminate instance {{(pid=124604) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:41:40.484430 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2e7c3e96-24c6-4feb-b158-c4f782597712 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "DELETE /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 204 len: 0 microversion: 2.1 time: 0.391848 May 28 03:41:40.485012 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 401/778] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:41:40 2022] DELETE /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 28 03:41:40.493498 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e793060a-21f4-4b53-a6b6-849be4a34a90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:40.496903 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e793060a-21f4-4b53-a6b6-849be4a34a90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:40.510824 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e793060a-21f4-4b53-a6b6-849be4a34a90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:40.511359 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e793060a-21f4-4b53-a6b6-849be4a34a90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:40.515340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e793060a-21f4-4b53-a6b6-849be4a34a90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:40.515340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e793060a-21f4-4b53-a6b6-849be4a34a90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:40.658765 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e793060a-21f4-4b53-a6b6-849be4a34a90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:40.669888 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e793060a-21f4-4b53-a6b6-849be4a34a90 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1689 microversion: 2.1 time: 0.180555 May 28 03:41:40.671481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 378/779] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:40 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1689 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:40.913935 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-4b9d7a16-1a9d-4934-bb51-bef48bc25453 req-30e69548-b650-49c7-8aa2-190d2c60ecef service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f65cf50d-baf3-4a38-9dec-14ff91d5f6ac", "name": "network-vif-plugged", "status": "completed", "tag": "b60c0498-d47b-4439-b615-1ff9bb9a032a"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:40.924795 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-4b9d7a16-1a9d-4934-bb51-bef48bc25453 req-30e69548-b650-49c7-8aa2-190d2c60ecef service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:40.929796 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-4b9d7a16-1a9d-4934-bb51-bef48bc25453 req-30e69548-b650-49c7-8aa2-190d2c60ecef service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:40.958617 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-4b9d7a16-1a9d-4934-bb51-bef48bc25453 req-30e69548-b650-49c7-8aa2-190d2c60ecef service nova] Creating event network-vif-plugged:b60c0498-d47b-4439-b615-1ff9bb9a032a for instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac on hv-cinder-84336 May 28 03:41:40.971686 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-4b9d7a16-1a9d-4934-bb51-bef48bc25453 req-30e69548-b650-49c7-8aa2-190d2c60ecef service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.218058 May 28 03:41:40.974565 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 402/780] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:41:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:41.159429 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-831ec7e6-ab53-4956-ae88-f839d4d5527a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:41.160108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-831ec7e6-ab53-4956-ae88-f839d4d5527a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:41.162828 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:41.168009 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-831ec7e6-ab53-4956-ae88-f839d4d5527a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:41.168466 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-831ec7e6-ab53-4956-ae88-f839d4d5527a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:41.169282 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-831ec7e6-ab53-4956-ae88-f839d4d5527a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:41.169721 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-831ec7e6-ab53-4956-ae88-f839d4d5527a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:41.310469 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-831ec7e6-ab53-4956-ae88-f839d4d5527a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:41.318636 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-831ec7e6-ab53-4956-ae88-f839d4d5527a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1367 microversion: 2.1 time: 0.163749 May 28 03:41:41.319212 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 379/781] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:41 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1367 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:41.684602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=6 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:41.689718 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-49135693-9275-4eca-8359-282100063aec tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:41.690947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-49135693-9275-4eca-8359-282100063aec tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:41.699227 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49135693-9275-4eca-8359-282100063aec tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:41.699722 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49135693-9275-4eca-8359-282100063aec tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:41.700560 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49135693-9275-4eca-8359-282100063aec tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:41.700995 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49135693-9275-4eca-8359-282100063aec tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:41.873779 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-49135693-9275-4eca-8359-282100063aec tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:41.882082 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-49135693-9275-4eca-8359-282100063aec tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1689 microversion: 2.1 time: 0.198494 May 28 03:41:41.882749 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 403/782] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:41 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1689 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:41.993198 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6e22e80c-d6dd-42fc-aa9f-a77515529f92 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:41.993703 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6e22e80c-d6dd-42fc-aa9f-a77515529f92 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:42.001213 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6e22e80c-d6dd-42fc-aa9f-a77515529f92 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:42.001873 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6e22e80c-d6dd-42fc-aa9f-a77515529f92 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:42.002867 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6e22e80c-d6dd-42fc-aa9f-a77515529f92 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:42.003304 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6e22e80c-d6dd-42fc-aa9f-a77515529f92 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:42.007286 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:42.030238 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6e22e80c-d6dd-42fc-aa9f-a77515529f92 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/ips" status: 200 len: 153 microversion: 2.20 time: 0.316225 May 28 03:41:42.030807 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 380/783] 10.222.0.41 () {60 vars in 1288 bytes} [Sat May 28 03:41:41 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/ips => generated 153 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:41:42.335917 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-089d9fa2-30bf-482e-8ca0-7b419e5c94a2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:42.336582 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-089d9fa2-30bf-482e-8ca0-7b419e5c94a2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:42.343893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-089d9fa2-30bf-482e-8ca0-7b419e5c94a2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:42.344331 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-089d9fa2-30bf-482e-8ca0-7b419e5c94a2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:42.345098 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-089d9fa2-30bf-482e-8ca0-7b419e5c94a2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:42.345556 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-089d9fa2-30bf-482e-8ca0-7b419e5c94a2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:42.498592 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-089d9fa2-30bf-482e-8ca0-7b419e5c94a2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:42.512916 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-089d9fa2-30bf-482e-8ca0-7b419e5c94a2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1367 microversion: 2.1 time: 0.180794 May 28 03:41:42.513527 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 404/784] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:42 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1367 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:42.899575 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-311a6ee4-c035-4c2b-8a57-8460c4439302 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:42.900343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-311a6ee4-c035-4c2b-8a57-8460c4439302 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:42.919497 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=8 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:42.923010 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-311a6ee4-c035-4c2b-8a57-8460c4439302 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:42.923659 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-311a6ee4-c035-4c2b-8a57-8460c4439302 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:42.924639 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-311a6ee4-c035-4c2b-8a57-8460c4439302 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:42.925256 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-311a6ee4-c035-4c2b-8a57-8460c4439302 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:43.157788 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-311a6ee4-c035-4c2b-8a57-8460c4439302 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:43.165561 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-311a6ee4-c035-4c2b-8a57-8460c4439302 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1689 microversion: 2.1 time: 0.269925 May 28 03:41:43.166358 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 381/785] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:42 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1689 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:43.537077 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-db8af830-6dfd-42fa-b449-bd43534f167a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:43.539345 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-db8af830-6dfd-42fa-b449-bd43534f167a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:43.548720 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-db8af830-6dfd-42fa-b449-bd43534f167a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:43.549431 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-db8af830-6dfd-42fa-b449-bd43534f167a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:43.550441 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-db8af830-6dfd-42fa-b449-bd43534f167a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:43.551043 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-db8af830-6dfd-42fa-b449-bd43534f167a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:43.573412 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:43.771624 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-db8af830-6dfd-42fa-b449-bd43534f167a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:43.802420 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-db8af830-6dfd-42fa-b449-bd43534f167a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1367 microversion: 2.1 time: 0.272786 May 28 03:41:43.803260 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 405/786] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:43 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1367 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:41:44.189969 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-281482a8-1d4c-4c77-b680-503fccada4af tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:44.192491 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-281482a8-1d4c-4c77-b680-503fccada4af tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:44.219721 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-281482a8-1d4c-4c77-b680-503fccada4af tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:44.220230 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-281482a8-1d4c-4c77-b680-503fccada4af tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:44.221037 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-281482a8-1d4c-4c77-b680-503fccada4af tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:44.221537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-281482a8-1d4c-4c77-b680-503fccada4af tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:44.235292 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5d39c14d-73cc-42b7-921f-242784ae41df tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:44.236066 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5d39c14d-73cc-42b7-921f-242784ae41df tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:44.252971 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5d39c14d-73cc-42b7-921f-242784ae41df tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:44.253737 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5d39c14d-73cc-42b7-921f-242784ae41df tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:44.254824 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5d39c14d-73cc-42b7-921f-242784ae41df tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:44.255645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5d39c14d-73cc-42b7-921f-242784ae41df tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:44.296475 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-5d39c14d-73cc-42b7-921f-242784ae41df tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lazy-loading 'flavor' on Instance uuid 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:41:44.774178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-281482a8-1d4c-4c77-b680-503fccada4af tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:44.787995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-281482a8-1d4c-4c77-b680-503fccada4af tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1689 microversion: 2.1 time: 0.609253 May 28 03:41:44.796532 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 382/787] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:44 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1689 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:44.827651 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9fdd34dd-b884-4f87-a8c2-0991b314c9b1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:44.834031 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9fdd34dd-b884-4f87-a8c2-0991b314c9b1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:44.842507 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9fdd34dd-b884-4f87-a8c2-0991b314c9b1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:44.842951 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9fdd34dd-b884-4f87-a8c2-0991b314c9b1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:44.843769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9fdd34dd-b884-4f87-a8c2-0991b314c9b1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:44.844147 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9fdd34dd-b884-4f87-a8c2-0991b314c9b1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:44.964335 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5d39c14d-73cc-42b7-921f-242784ae41df tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "POST /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/action" status: 202 len: 0 microversion: 2.20 time: 0.733321 May 28 03:41:44.967226 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 406/788] 10.222.0.41 () {62 vars in 1318 bytes} [Sat May 28 03:41:44 2022] POST /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/action => generated 0 bytes in 737 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 28 03:41:44.975433 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-753d9b39-721c-41ff-b87c-0b78ee91d6b9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:44.976378 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-753d9b39-721c-41ff-b87c-0b78ee91d6b9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:44.983774 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-753d9b39-721c-41ff-b87c-0b78ee91d6b9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:44.985811 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-753d9b39-721c-41ff-b87c-0b78ee91d6b9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:44.995563 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-753d9b39-721c-41ff-b87c-0b78ee91d6b9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:44.997025 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-753d9b39-721c-41ff-b87c-0b78ee91d6b9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:45.191824 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-753d9b39-721c-41ff-b87c-0b78ee91d6b9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:45.212129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-753d9b39-721c-41ff-b87c-0b78ee91d6b9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:45.216120 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-753d9b39-721c-41ff-b87c-0b78ee91d6b9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:45.216120 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-753d9b39-721c-41ff-b87c-0b78ee91d6b9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.245717 May 28 03:41:45.216120 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 407/789] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:44 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:45.504205 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9fdd34dd-b884-4f87-a8c2-0991b314c9b1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:45.512063 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9fdd34dd-b884-4f87-a8c2-0991b314c9b1 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1367 microversion: 2.1 time: 0.688737 May 28 03:41:45.512944 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 383/790] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:44 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1367 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:45.805755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-32fda35f-5e4f-4694-81c0-70d00ffbe3b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:45.806363 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-32fda35f-5e4f-4694-81c0-70d00ffbe3b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:45.813056 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-32fda35f-5e4f-4694-81c0-70d00ffbe3b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:45.813786 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-32fda35f-5e4f-4694-81c0-70d00ffbe3b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:45.814538 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-32fda35f-5e4f-4694-81c0-70d00ffbe3b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:45.814971 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-32fda35f-5e4f-4694-81c0-70d00ffbe3b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:45.900876 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-32fda35f-5e4f-4694-81c0-70d00ffbe3b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:45.909445 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-32fda35f-5e4f-4694-81c0-70d00ffbe3b6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1621 microversion: 2.1 time: 0.107518 May 28 03:41:45.909764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 408/791] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:45 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1621 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:46.164368 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-58038577-3ff1-47a4-99e8-df8c71538f0f req-d6ef9ba4-8237-49d4-aacd-1486456f8573 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690", "name": "network-vif-deleted", "tag": "ea42d3d0-51a0-41a4-b165-b66f57c57a62"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:46.173475 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-58038577-3ff1-47a4-99e8-df8c71538f0f req-d6ef9ba4-8237-49d4-aacd-1486456f8573 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:46.173920 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-58038577-3ff1-47a4-99e8-df8c71538f0f req-d6ef9ba4-8237-49d4-aacd-1486456f8573 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:46.186064 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-58038577-3ff1-47a4-99e8-df8c71538f0f req-d6ef9ba4-8237-49d4-aacd-1486456f8573 service nova] Creating event network-vif-deleted:ea42d3d0-51a0-41a4-b165-b66f57c57a62 for instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 on hv-cinder-84336 May 28 03:41:46.198185 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-58038577-3ff1-47a4-99e8-df8c71538f0f req-d6ef9ba4-8237-49d4-aacd-1486456f8573 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037692 May 28 03:41:46.199041 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 384/792] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:41:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:46.235341 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-503bcb54-9139-4b04-9aa3-15d3e8376ac2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:46.236095 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-503bcb54-9139-4b04-9aa3-15d3e8376ac2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:46.243455 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-503bcb54-9139-4b04-9aa3-15d3e8376ac2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:46.244274 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-503bcb54-9139-4b04-9aa3-15d3e8376ac2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:46.245486 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-503bcb54-9139-4b04-9aa3-15d3e8376ac2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:46.246110 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-503bcb54-9139-4b04-9aa3-15d3e8376ac2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:46.416720 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-503bcb54-9139-4b04-9aa3-15d3e8376ac2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:46.427520 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-503bcb54-9139-4b04-9aa3-15d3e8376ac2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:46.429750 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-503bcb54-9139-4b04-9aa3-15d3e8376ac2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:46.431361 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-503bcb54-9139-4b04-9aa3-15d3e8376ac2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.201243 May 28 03:41:46.444959 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 409/793] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:46 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:46.531960 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-bf314ee4-ae4d-499f-9118-8456bab52d8b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:46.533439 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-bf314ee4-ae4d-499f-9118-8456bab52d8b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:46.540001 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bf314ee4-ae4d-499f-9118-8456bab52d8b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:46.540688 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bf314ee4-ae4d-499f-9118-8456bab52d8b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:46.541740 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bf314ee4-ae4d-499f-9118-8456bab52d8b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:46.542334 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bf314ee4-ae4d-499f-9118-8456bab52d8b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:46.800722 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-bf314ee4-ae4d-499f-9118-8456bab52d8b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:46.811690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-bf314ee4-ae4d-499f-9118-8456bab52d8b tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1557 microversion: 2.1 time: 0.284932 May 28 03:41:46.813626 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 385/794] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:46 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1557 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:46.829949 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-cd5fdd8d-dbc9-45b2-8bf7-4d6552ef8075 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:46.831835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-cd5fdd8d-dbc9-45b2-8bf7-4d6552ef8075 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:46.838259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cd5fdd8d-dbc9-45b2-8bf7-4d6552ef8075 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:46.842129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cd5fdd8d-dbc9-45b2-8bf7-4d6552ef8075 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:46.842129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cd5fdd8d-dbc9-45b2-8bf7-4d6552ef8075 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:46.842129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cd5fdd8d-dbc9-45b2-8bf7-4d6552ef8075 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:46.930708 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-786c0c91-37ab-406b-b6f6-cda23c26393d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:46.932908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-786c0c91-37ab-406b-b6f6-cda23c26393d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:46.956648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-786c0c91-37ab-406b-b6f6-cda23c26393d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:46.957421 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-786c0c91-37ab-406b-b6f6-cda23c26393d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:46.958515 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-786c0c91-37ab-406b-b6f6-cda23c26393d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:46.961309 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-786c0c91-37ab-406b-b6f6-cda23c26393d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:46.976467 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-cd5fdd8d-dbc9-45b2-8bf7-4d6552ef8075 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.150396 May 28 03:41:46.977753 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 410/795] 10.222.0.41 () {58 vars in 1274 bytes} [Sat May 28 03:41:46 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/os-interface => generated 285 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:47.031326 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-786c0c91-37ab-406b-b6f6-cda23c26393d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:47.031782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-786c0c91-37ab-406b-b6f6-cda23c26393d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:47.032240 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-786c0c91-37ab-406b-b6f6-cda23c26393d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lazy-loading 'fault' on Instance uuid 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:41:47.159970 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-786c0c91-37ab-406b-b6f6-cda23c26393d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:47.175195 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-786c0c91-37ab-406b-b6f6-cda23c26393d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 200 len: 1335 microversion: 2.1 time: 0.250008 May 28 03:41:47.181947 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 386/796] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:46 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 1335 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:47.455234 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1751a19a-3353-4811-b2c4-8a1442a7cce2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:47.455671 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1751a19a-3353-4811-b2c4-8a1442a7cce2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:47.461930 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1751a19a-3353-4811-b2c4-8a1442a7cce2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:47.462375 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1751a19a-3353-4811-b2c4-8a1442a7cce2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:47.463150 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1751a19a-3353-4811-b2c4-8a1442a7cce2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:47.463579 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1751a19a-3353-4811-b2c4-8a1442a7cce2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:47.745400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1751a19a-3353-4811-b2c4-8a1442a7cce2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:47.758379 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1751a19a-3353-4811-b2c4-8a1442a7cce2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:47.763731 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1751a19a-3353-4811-b2c4-8a1442a7cce2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:47.768855 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1751a19a-3353-4811-b2c4-8a1442a7cce2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.318310 May 28 03:41:47.769663 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 411/797] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:47 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:48.199761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-528bc08c-2af3-4d75-b041-4030f45112c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:48.203714 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-528bc08c-2af3-4d75-b041-4030f45112c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:48.211963 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-528bc08c-2af3-4d75-b041-4030f45112c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:48.212420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-528bc08c-2af3-4d75-b041-4030f45112c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:48.213429 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-528bc08c-2af3-4d75-b041-4030f45112c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:48.213929 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-528bc08c-2af3-4d75-b041-4030f45112c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:48.242671 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-528bc08c-2af3-4d75-b041-4030f45112c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] HTTP exception thrown: Instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 could not be found. May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-528bc08c-2af3-4d75-b041-4030f45112c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Request method failure captured: May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 HTTP/1.1 May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZioFSMjchJLpIZqod5q-7lgl8fEY36gE52nylmpmtlBCMGq9sB1-y2xrNX1ntFqjnFW5i92fC2liYIvh0AHl9bCuVQTnAqKQCbialBgg6tn7xuEEa8ZKcsvKvenrdvzcDN3coDLFL7-jztA0Tb3mMeshJqy19Ht6r4OU5cn2edXS_E May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 5cab8255c6134aebb9c93493a2342cb1 May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:41:48.254924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 5cab8255c6134aebb9c93493a2342cb1 May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-ServerRescueNegativeTestJSON-968064324-project May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: b6709eed72ca46468fd298929ae0f764 May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-ServerRescueNegativeTestJSON-968064324-project May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 could not be found. May 28 03:41:48.256351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690'} May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 could not be found. May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:41:48.264850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 could not be found. May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 could not be found. May 28 03:41:48.266593 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:41:48.268903 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-528bc08c-2af3-4d75-b041-4030f45112c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Returning 404 to user: Instance 13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:41:48.268903 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-528bc08c-2af3-4d75-b041-4030f45112c0 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690" status: 404 len: 111 microversion: 2.1 time: 0.060429 May 28 03:41:48.268903 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 387/798] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:48 2022] GET /compute/v2.1/servers/13c3d7e5-a696-4d8c-8d9c-1d4fdd2cd690 => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:41:48.272426 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e3bc6d47-fe25-4b1d-80f6-54b455c18293 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '.version_select at 0x7feb2c156820>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:48.278009 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:48.348797 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-7a74a92a-95bf-48b7-a7b6-302d5f9e9584 req-f1303c35-16c4-404c-865b-3ee87a1d4abd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f65cf50d-baf3-4a38-9dec-14ff91d5f6ac", "tag": "b60c0498-d47b-4439-b615-1ff9bb9a032a"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:48.352617 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=72,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:48.361079 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-7a74a92a-95bf-48b7-a7b6-302d5f9e9584 req-f1303c35-16c4-404c-865b-3ee87a1d4abd service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:48.361543 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-7a74a92a-95bf-48b7-a7b6-302d5f9e9584 req-f1303c35-16c4-404c-865b-3ee87a1d4abd service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:48.375783 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-7a74a92a-95bf-48b7-a7b6-302d5f9e9584 req-f1303c35-16c4-404c-865b-3ee87a1d4abd service nova] Creating event network-changed:b60c0498-d47b-4439-b615-1ff9bb9a032a for instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac on hv-cinder-84336 May 28 03:41:48.382993 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e3bc6d47-fe25-4b1d-80f6-54b455c18293 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-133479696" status: 202 len: 0 microversion: 2.1 time: 0.116762 May 28 03:41:48.383898 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 412/799] 10.222.0.41 () {58 vars in 1217 bytes} [Sat May 28 03:41:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-133479696 => generated 0 bytes in 118 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:41:48.390511 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-7a74a92a-95bf-48b7-a7b6-302d5f9e9584 req-f1303c35-16c4-404c-865b-3ee87a1d4abd service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059697 May 28 03:41:48.391554 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 388/800] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:41:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:41:48.788545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c08f5725-593d-4d5c-83ce-9c1e35caf716 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:48.790286 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c08f5725-593d-4d5c-83ce-9c1e35caf716 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:48.799778 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c08f5725-593d-4d5c-83ce-9c1e35caf716 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:48.800605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c08f5725-593d-4d5c-83ce-9c1e35caf716 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:48.801888 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c08f5725-593d-4d5c-83ce-9c1e35caf716 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:48.802567 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c08f5725-593d-4d5c-83ce-9c1e35caf716 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:49.115671 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c08f5725-593d-4d5c-83ce-9c1e35caf716 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:49.126038 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c08f5725-593d-4d5c-83ce-9c1e35caf716 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:49.128531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c08f5725-593d-4d5c-83ce-9c1e35caf716 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:49.131330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c08f5725-593d-4d5c-83ce-9c1e35caf716 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.347526 May 28 03:41:49.143416 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 413/801] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:48 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:49.457848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c30fc459-4c47-438c-a347-95e9f23a4ee5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Action: 'create', calling method: .version_select at 0x7feb2c044af0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1597361276", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOqk2aIyV9SiB4dDC9oJkoAjaegbTpQ08/x3PxRqlOMlj4k+c1jVfL8MuFF4R72d4YasHcBtJd6dIteKlmNc7N7va1gdD8Wqf17UD48/FK7BA58CuSNjZhMrMOM3VxKw5w=="}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:49.473788 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-c30fc459-4c47-438c-a347-95e9f23a4ee5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Getting quotas for project c0fe831f6e954e05b7d448f1d4f43754. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:41:49.478463 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-c30fc459-4c47-438c-a347-95e9f23a4ee5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Getting quotas for user 2b515cebf18643c38f1cfc2e3fad1db4 and project c0fe831f6e954e05b7d448f1d4f43754. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:41:49.560373 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c30fc459-4c47-438c-a347-95e9f23a4ee5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.242151 May 28 03:41:49.560972 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 389/802] 10.222.0.41 () {60 vars in 1158 bytes} [Sat May 28 03:41:49 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:50.150193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4d15fb59-9592-498a-9489-825f430caba3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:50.150809 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4d15fb59-9592-498a-9489-825f430caba3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:50.157432 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4d15fb59-9592-498a-9489-825f430caba3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:50.157885 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4d15fb59-9592-498a-9489-825f430caba3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:50.158663 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4d15fb59-9592-498a-9489-825f430caba3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:50.159104 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4d15fb59-9592-498a-9489-825f430caba3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:50.303655 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4d15fb59-9592-498a-9489-825f430caba3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:50.312791 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4d15fb59-9592-498a-9489-825f430caba3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:50.314089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4d15fb59-9592-498a-9489-825f430caba3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:50.331192 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4d15fb59-9592-498a-9489-825f430caba3 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.168995 May 28 03:41:50.331192 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 414/803] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:50 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:51.346667 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a495178c-92a0-4f2d-ae3d-9bbb047223d9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:51.346667 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a495178c-92a0-4f2d-ae3d-9bbb047223d9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:51.346667 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a495178c-92a0-4f2d-ae3d-9bbb047223d9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:51.346667 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a495178c-92a0-4f2d-ae3d-9bbb047223d9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:51.347406 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a495178c-92a0-4f2d-ae3d-9bbb047223d9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:51.347406 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a495178c-92a0-4f2d-ae3d-9bbb047223d9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:51.504854 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a495178c-92a0-4f2d-ae3d-9bbb047223d9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:51.518961 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a495178c-92a0-4f2d-ae3d-9bbb047223d9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:51.520784 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a495178c-92a0-4f2d-ae3d-9bbb047223d9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:51.522295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a495178c-92a0-4f2d-ae3d-9bbb047223d9 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.193624 May 28 03:41:51.523011 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 390/804] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:51 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:51.952954 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a4c2d2f5-5d6c-48db-9da0-78a9e186367b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:51.953569 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a4c2d2f5-5d6c-48db-9da0-78a9e186367b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:51.960315 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a4c2d2f5-5d6c-48db-9da0-78a9e186367b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:51.961281 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a4c2d2f5-5d6c-48db-9da0-78a9e186367b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:51.962177 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a4c2d2f5-5d6c-48db-9da0-78a9e186367b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:51.962624 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a4c2d2f5-5d6c-48db-9da0-78a9e186367b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:51.988091 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a4c2d2f5-5d6c-48db-9da0-78a9e186367b tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/ips" status: 200 len: 146 microversion: 2.1 time: 0.039066 May 28 03:41:51.988664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 415/805] 10.222.0.41 () {58 vars in 1247 bytes} [Sat May 28 03:41:51 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/ips => generated 146 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:52.543847 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-85b7d433-9d96-46f8-b3f3-4a8eb3f1969d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:52.544518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-85b7d433-9d96-46f8-b3f3-4a8eb3f1969d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:52.551538 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-85b7d433-9d96-46f8-b3f3-4a8eb3f1969d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:52.551993 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-85b7d433-9d96-46f8-b3f3-4a8eb3f1969d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:52.552761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-85b7d433-9d96-46f8-b3f3-4a8eb3f1969d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:52.553268 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-85b7d433-9d96-46f8-b3f3-4a8eb3f1969d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:52.714791 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-85b7d433-9d96-46f8-b3f3-4a8eb3f1969d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:52.731535 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-85b7d433-9d96-46f8-b3f3-4a8eb3f1969d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:52.733344 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-85b7d433-9d96-46f8-b3f3-4a8eb3f1969d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:52.738279 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-85b7d433-9d96-46f8-b3f3-4a8eb3f1969d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.198451 May 28 03:41:52.740087 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 391/806] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:52 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:52.768524 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2eec49d4-0a8f-4cd2-92b2-f433b60f93bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:52.769389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2eec49d4-0a8f-4cd2-92b2-f433b60f93bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:52.777166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2eec49d4-0a8f-4cd2-92b2-f433b60f93bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:52.777366 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2eec49d4-0a8f-4cd2-92b2-f433b60f93bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:52.778149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2eec49d4-0a8f-4cd2-92b2-f433b60f93bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:52.778655 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2eec49d4-0a8f-4cd2-92b2-f433b60f93bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:52.808097 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2eec49d4-0a8f-4cd2-92b2-f433b60f93bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Going to try to terminate instance {{(pid=124604) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:41:53.175668 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2eec49d4-0a8f-4cd2-92b2-f433b60f93bd tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "DELETE /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 204 len: 0 microversion: 2.1 time: 0.410714 May 28 03:41:53.185832 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 416/807] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:41:52 2022] DELETE /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 28 03:41:53.193117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-497f4c25-b386-4988-8351-2a66fa5b5b59 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:53.193762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-497f4c25-b386-4988-8351-2a66fa5b5b59 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:53.199811 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-497f4c25-b386-4988-8351-2a66fa5b5b59 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:53.200257 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-497f4c25-b386-4988-8351-2a66fa5b5b59 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:53.201323 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-497f4c25-b386-4988-8351-2a66fa5b5b59 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:53.201727 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-497f4c25-b386-4988-8351-2a66fa5b5b59 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:53.380609 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-497f4c25-b386-4988-8351-2a66fa5b5b59 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:53.388554 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-497f4c25-b386-4988-8351-2a66fa5b5b59 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1509 microversion: 2.1 time: 0.199193 May 28 03:41:53.389384 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 392/808] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:53 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1509 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:53.763321 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dbacb294-7353-4f4b-9c30-eddb6438f31e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:53.764138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dbacb294-7353-4f4b-9c30-eddb6438f31e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:53.770740 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dbacb294-7353-4f4b-9c30-eddb6438f31e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:53.771610 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dbacb294-7353-4f4b-9c30-eddb6438f31e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:53.772514 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dbacb294-7353-4f4b-9c30-eddb6438f31e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:53.773051 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dbacb294-7353-4f4b-9c30-eddb6438f31e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:53.902879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6f3b2145-d872-4159-9dce-2f7614c622bc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "43e258a6-edd7-4013-bbba-387dc62c7053"}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:53.904195 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6f3b2145-d872-4159-9dce-2f7614c622bc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:53.918923 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f3b2145-d872-4159-9dce-2f7614c622bc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:53.919559 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f3b2145-d872-4159-9dce-2f7614c622bc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:53.920910 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f3b2145-d872-4159-9dce-2f7614c622bc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:53.921570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6f3b2145-d872-4159-9dce-2f7614c622bc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:53.936425 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dbacb294-7353-4f4b-9c30-eddb6438f31e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:53.945131 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dbacb294-7353-4f4b-9c30-eddb6438f31e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:53.946554 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dbacb294-7353-4f4b-9c30-eddb6438f31e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:53.947793 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-dbacb294-7353-4f4b-9c30-eddb6438f31e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.188816 May 28 03:41:53.948338 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 417/809] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:53 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:54.407068 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-af08dc99-8ff6-4753-97b2-34cbbdbbe214 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:54.407760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-af08dc99-8ff6-4753-97b2-34cbbdbbe214 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:54.414910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af08dc99-8ff6-4753-97b2-34cbbdbbe214 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:54.415358 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af08dc99-8ff6-4753-97b2-34cbbdbbe214 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:54.416328 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af08dc99-8ff6-4753-97b2-34cbbdbbe214 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:54.416961 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af08dc99-8ff6-4753-97b2-34cbbdbbe214 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:54.562177 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-af08dc99-8ff6-4753-97b2-34cbbdbbe214 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:54.572789 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-af08dc99-8ff6-4753-97b2-34cbbdbbe214 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1509 microversion: 2.1 time: 0.170735 May 28 03:41:54.573387 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 418/810] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:54 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1509 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:41:54.934258 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6f3b2145-d872-4159-9dce-2f7614c622bc tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "POST /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.036323 May 28 03:41:54.935155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 393/811] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:41:53 2022] POST /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments => generated 194 bytes in 1038 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:54.968217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-06ab24be-076d-4a32-a65c-c8bbec573eb5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:54.968977 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-06ab24be-076d-4a32-a65c-c8bbec573eb5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:54.976259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06ab24be-076d-4a32-a65c-c8bbec573eb5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:54.981567 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06ab24be-076d-4a32-a65c-c8bbec573eb5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:54.981567 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06ab24be-076d-4a32-a65c-c8bbec573eb5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:54.981567 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06ab24be-076d-4a32-a65c-c8bbec573eb5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:55.147556 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-06ab24be-076d-4a32-a65c-c8bbec573eb5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:55.156780 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-06ab24be-076d-4a32-a65c-c8bbec573eb5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:55.158531 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-06ab24be-076d-4a32-a65c-c8bbec573eb5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:55.171568 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-06ab24be-076d-4a32-a65c-c8bbec573eb5 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.197081 May 28 03:41:55.171568 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 419/812] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:54 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:55.476225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-b3c25662-fdde-47db-83a3-bccf271c7536 req-6d3922d7-c8f6-44b7-a48f-7bc403a70ccb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "247082dc-5284-462d-92c8-045154d864e4", "name": "network-vif-deleted", "tag": "6aaa9bcc-6545-4d6b-95af-ee421ef5771b"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:55.485098 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-b3c25662-fdde-47db-83a3-bccf271c7536 req-6d3922d7-c8f6-44b7-a48f-7bc403a70ccb service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:55.485715 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-b3c25662-fdde-47db-83a3-bccf271c7536 req-6d3922d7-c8f6-44b7-a48f-7bc403a70ccb service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:55.498031 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-b3c25662-fdde-47db-83a3-bccf271c7536 req-6d3922d7-c8f6-44b7-a48f-7bc403a70ccb service nova] Creating event network-vif-deleted:6aaa9bcc-6545-4d6b-95af-ee421ef5771b for instance 247082dc-5284-462d-92c8-045154d864e4 on hv-cinder-84336 May 28 03:41:55.503378 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-b3c25662-fdde-47db-83a3-bccf271c7536 req-6d3922d7-c8f6-44b7-a48f-7bc403a70ccb service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031726 May 28 03:41:55.503948 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 394/813] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:41:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:41:55.590867 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-73abfeb8-0662-48fc-9552-1434a81443e6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:55.591484 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-73abfeb8-0662-48fc-9552-1434a81443e6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:55.599168 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-73abfeb8-0662-48fc-9552-1434a81443e6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:55.599693 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-73abfeb8-0662-48fc-9552-1434a81443e6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:55.601424 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-73abfeb8-0662-48fc-9552-1434a81443e6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:55.601872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-73abfeb8-0662-48fc-9552-1434a81443e6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:55.610163 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:55.667757 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-73abfeb8-0662-48fc-9552-1434a81443e6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:55.675995 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-73abfeb8-0662-48fc-9552-1434a81443e6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1467 microversion: 2.1 time: 0.089083 May 28 03:41:55.676473 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 420/814] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:55 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1467 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:56.181476 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d21a8897-e032-4701-9552-a0ece2954584 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:56.182181 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d21a8897-e032-4701-9552-a0ece2954584 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:56.189490 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d21a8897-e032-4701-9552-a0ece2954584 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:56.190058 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d21a8897-e032-4701-9552-a0ece2954584 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:56.190821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d21a8897-e032-4701-9552-a0ece2954584 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:56.191256 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d21a8897-e032-4701-9552-a0ece2954584 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:56.335439 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d21a8897-e032-4701-9552-a0ece2954584 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:56.344874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d21a8897-e032-4701-9552-a0ece2954584 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:56.346950 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d21a8897-e032-4701-9552-a0ece2954584 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:56.348701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d21a8897-e032-4701-9552-a0ece2954584 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.172981 May 28 03:41:56.349472 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 395/815] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:56 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:56.693676 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b3ecc64f-9723-4c6b-987b-4817d110bad6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:56.694292 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b3ecc64f-9723-4c6b-987b-4817d110bad6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:56.701345 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3ecc64f-9723-4c6b-987b-4817d110bad6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:56.701807 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3ecc64f-9723-4c6b-987b-4817d110bad6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:56.702600 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3ecc64f-9723-4c6b-987b-4817d110bad6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:56.703030 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3ecc64f-9723-4c6b-987b-4817d110bad6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:56.735118 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3ecc64f-9723-4c6b-987b-4817d110bad6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:56.735569 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b3ecc64f-9723-4c6b-987b-4817d110bad6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:56.736018 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-b3ecc64f-9723-4c6b-987b-4817d110bad6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lazy-loading 'fault' on Instance uuid 247082dc-5284-462d-92c8-045154d864e4 {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:41:56.780350 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b3ecc64f-9723-4c6b-987b-4817d110bad6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:56.788099 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b3ecc64f-9723-4c6b-987b-4817d110bad6 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 200 len: 1312 microversion: 2.1 time: 0.098734 May 28 03:41:56.788709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 421/816] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:56 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 1312 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:57.367075 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c6c9f1c2-487b-4bb3-9292-4b265ecd09c1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:57.367688 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c6c9f1c2-487b-4bb3-9292-4b265ecd09c1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:57.375648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c6c9f1c2-487b-4bb3-9292-4b265ecd09c1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:57.376094 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c6c9f1c2-487b-4bb3-9292-4b265ecd09c1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:57.377479 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c6c9f1c2-487b-4bb3-9292-4b265ecd09c1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:57.377983 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c6c9f1c2-487b-4bb3-9292-4b265ecd09c1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:57.507879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c6c9f1c2-487b-4bb3-9292-4b265ecd09c1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:57.515878 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c6c9f1c2-487b-4bb3-9292-4b265ecd09c1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:57.517397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c6c9f1c2-487b-4bb3-9292-4b265ecd09c1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:57.519143 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c6c9f1c2-487b-4bb3-9292-4b265ecd09c1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1745 microversion: 2.20 time: 0.156055 May 28 03:41:57.519693 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 396/817] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:57 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1745 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:57.805829 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5d31d5a7-ce13-425b-a38b-7b519c33a87d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:57.806460 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5d31d5a7-ce13-425b-a38b-7b519c33a87d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 247082dc-5284-462d-92c8-045154d864e4] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:57.813319 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5d31d5a7-ce13-425b-a38b-7b519c33a87d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:57.813769 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5d31d5a7-ce13-425b-a38b-7b519c33a87d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:57.815163 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5d31d5a7-ce13-425b-a38b-7b519c33a87d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:57.815601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5d31d5a7-ce13-425b-a38b-7b519c33a87d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:57.829507 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-5d31d5a7-ce13-425b-a38b-7b519c33a87d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] HTTP exception thrown: Instance 247082dc-5284-462d-92c8-045154d864e4 could not be found. May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5d31d5a7-ce13-425b-a38b-7b519c33a87d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Request method failure captured: May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 HTTP/1.1 May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZioFSMjchJLpIZqod5q-7lgl8fEY36gE52nylmpmtlBCMGq9sB1-y2xrNX1ntFqjnFW5i92fC2liYIvh0AHl9bCuVQTnAqKQCbialBgg6tn7xuEEa8ZKcsvKvenrdvzcDN3coDLFL7-jztA0Tb3mMeshJqy19Ht6r4OU5cn2edXS_E May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 5cab8255c6134aebb9c93493a2342cb1 May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:41:57.831259 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 5cab8255c6134aebb9c93493a2342cb1 May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-ServerRescueNegativeTestJSON-968064324-project May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: b6709eed72ca46468fd298929ae0f764 May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-ServerRescueNegativeTestJSON-968064324-project May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance 247082dc-5284-462d-92c8-045154d864e4 could not be found. May 28 03:41:57.832837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': '247082dc-5284-462d-92c8-045154d864e4'} May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 247082dc-5284-462d-92c8-045154d864e4 could not be found. May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:41:57.833770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 247082dc-5284-462d-92c8-045154d864e4 could not be found. May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 247082dc-5284-462d-92c8-045154d864e4 could not be found. May 28 03:41:57.835476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:41:57.836992 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5d31d5a7-ce13-425b-a38b-7b519c33a87d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Returning 404 to user: Instance 247082dc-5284-462d-92c8-045154d864e4 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:41:57.836992 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5d31d5a7-ce13-425b-a38b-7b519c33a87d tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4" status: 404 len: 111 microversion: 2.1 time: 0.031017 May 28 03:41:57.836992 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 422/818] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:57 2022] GET /compute/v2.1/servers/247082dc-5284-462d-92c8-045154d864e4 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:41:57.841080 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f5125821-6228-4766-a99b-449769fd51c3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:57.841598 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f5125821-6228-4766-a99b-449769fd51c3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:57.847573 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f5125821-6228-4766-a99b-449769fd51c3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:57.848015 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f5125821-6228-4766-a99b-449769fd51c3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:57.848768 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f5125821-6228-4766-a99b-449769fd51c3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:57.849239 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f5125821-6228-4766-a99b-449769fd51c3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:57.874750 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f5125821-6228-4766-a99b-449769fd51c3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:41:57.987595 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:58.113384 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f5125821-6228-4766-a99b-449769fd51c3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "DELETE /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 204 len: 0 microversion: 2.1 time: 0.275705 May 28 03:41:58.114119 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 397/819] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:41:57 2022] DELETE /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:41:58.122125 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-54c79f5f-0b0c-4cf5-a0b4-132cd18d0321 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:58.122731 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-54c79f5f-0b0c-4cf5-a0b4-132cd18d0321 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:58.129649 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-54c79f5f-0b0c-4cf5-a0b4-132cd18d0321 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:58.130098 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-54c79f5f-0b0c-4cf5-a0b4-132cd18d0321 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:58.130906 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-54c79f5f-0b0c-4cf5-a0b4-132cd18d0321 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:58.131512 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-54c79f5f-0b0c-4cf5-a0b4-132cd18d0321 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:58.262903 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-54c79f5f-0b0c-4cf5-a0b4-132cd18d0321 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:58.270386 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-54c79f5f-0b0c-4cf5-a0b4-132cd18d0321 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.152297 May 28 03:41:58.270945 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 423/820] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:58 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:58.538975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3eeb1460-9779-43fb-81c7-7286c2548048 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:58.539676 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3eeb1460-9779-43fb-81c7-7286c2548048 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:58.546326 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3eeb1460-9779-43fb-81c7-7286c2548048 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:58.546726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3eeb1460-9779-43fb-81c7-7286c2548048 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:58.547503 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3eeb1460-9779-43fb-81c7-7286c2548048 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:58.547945 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3eeb1460-9779-43fb-81c7-7286c2548048 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:58.638326 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-3d38abeb-14a5-4b0d-ad2e-b9fbf1c5276c req-abfdf6c8-1ef2-454d-8b75-c469af76653e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6be2aeaa-8d90-4e51-87d2-d02a8566ac1a", "name": "network-vif-plugged", "status": "completed", "tag": "73f87b73-9d7c-43a7-9bd2-dc7fb5450112"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:41:58.649803 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-3d38abeb-14a5-4b0d-ad2e-b9fbf1c5276c req-abfdf6c8-1ef2-454d-8b75-c469af76653e service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:58.650275 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-3d38abeb-14a5-4b0d-ad2e-b9fbf1c5276c req-abfdf6c8-1ef2-454d-8b75-c469af76653e service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:58.662496 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-3d38abeb-14a5-4b0d-ad2e-b9fbf1c5276c req-abfdf6c8-1ef2-454d-8b75-c469af76653e service nova] Creating event network-vif-plugged:73f87b73-9d7c-43a7-9bd2-dc7fb5450112 for instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a on hv-cinder-84336 May 28 03:41:58.667585 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-3d38abeb-14a5-4b0d-ad2e-b9fbf1c5276c req-abfdf6c8-1ef2-454d-8b75-c469af76653e service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042386 May 28 03:41:58.668290 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 424/821] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:41:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:41:58.690963 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3eeb1460-9779-43fb-81c7-7286c2548048 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:58.698984 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3eeb1460-9779-43fb-81c7-7286c2548048 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:58.700492 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3eeb1460-9779-43fb-81c7-7286c2548048 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:58.701691 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3eeb1460-9779-43fb-81c7-7286c2548048 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1745 microversion: 2.20 time: 0.167400 May 28 03:41:58.702247 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 398/822] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:58 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1745 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:41:59.288375 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ceccf007-d865-478f-a0fa-857e9e1185b3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:59.289069 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ceccf007-d865-478f-a0fa-857e9e1185b3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:59.295244 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ceccf007-d865-478f-a0fa-857e9e1185b3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:59.295697 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ceccf007-d865-478f-a0fa-857e9e1185b3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:59.296463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ceccf007-d865-478f-a0fa-857e9e1185b3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:59.296902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ceccf007-d865-478f-a0fa-857e9e1185b3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:59.437049 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ceccf007-d865-478f-a0fa-857e9e1185b3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:59.445895 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ceccf007-d865-478f-a0fa-857e9e1185b3 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.162190 May 28 03:41:59.446596 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 425/823] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:41:59 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:41:59.720392 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5de9ddbc-8ae3-4e4e-8271-0bae23bfb7c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:41:59.720911 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5de9ddbc-8ae3-4e4e-8271-0bae23bfb7c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:41:59.723438 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=4 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:59.728732 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5de9ddbc-8ae3-4e4e-8271-0bae23bfb7c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:59.729220 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5de9ddbc-8ae3-4e4e-8271-0bae23bfb7c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:59.729994 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5de9ddbc-8ae3-4e4e-8271-0bae23bfb7c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:41:59.730447 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5de9ddbc-8ae3-4e4e-8271-0bae23bfb7c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:41:59.869537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5de9ddbc-8ae3-4e4e-8271-0bae23bfb7c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:59.877610 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5de9ddbc-8ae3-4e4e-8271-0bae23bfb7c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:59.879186 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5de9ddbc-8ae3-4e4e-8271-0bae23bfb7c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:41:59.880343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5de9ddbc-8ae3-4e4e-8271-0bae23bfb7c6 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1745 microversion: 2.20 time: 0.164065 May 28 03:41:59.880897 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 399/824] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:41:59 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1745 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:00.463370 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d13bb3fa-ce1f-48a8-b745-ffe3ef866ada tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:00.463986 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d13bb3fa-ce1f-48a8-b745-ffe3ef866ada tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:00.471399 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d13bb3fa-ce1f-48a8-b745-ffe3ef866ada tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:00.472109 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d13bb3fa-ce1f-48a8-b745-ffe3ef866ada tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:00.473070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d13bb3fa-ce1f-48a8-b745-ffe3ef866ada tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:00.473542 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d13bb3fa-ce1f-48a8-b745-ffe3ef866ada tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:00.612119 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d13bb3fa-ce1f-48a8-b745-ffe3ef866ada tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:00.622251 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d13bb3fa-ce1f-48a8-b745-ffe3ef866ada tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1522 microversion: 2.1 time: 0.162836 May 28 03:42:00.622883 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 426/825] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:00 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:00.898310 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0db47985-25f1-4320-9a29-696db4b5d3eb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:00.898927 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0db47985-25f1-4320-9a29-696db4b5d3eb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:00.919028 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0db47985-25f1-4320-9a29-696db4b5d3eb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:00.919665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0db47985-25f1-4320-9a29-696db4b5d3eb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:00.920495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0db47985-25f1-4320-9a29-696db4b5d3eb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:00.920948 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0db47985-25f1-4320-9a29-696db4b5d3eb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:01.102373 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0db47985-25f1-4320-9a29-696db4b5d3eb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:01.110797 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0db47985-25f1-4320-9a29-696db4b5d3eb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:01.112668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0db47985-25f1-4320-9a29-696db4b5d3eb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:01.113917 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0db47985-25f1-4320-9a29-696db4b5d3eb tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1740 microversion: 2.20 time: 0.220043 May 28 03:42:01.117134 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 400/826] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:00 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1740 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:01.239927 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-9e1524df-bc1c-4fcf-96b2-1bdf04984662 req-7707227e-d3e8-4095-bea1-024025d8c0a6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b4a0f56-eb59-4f13-a90a-442b9a2e0af8", "name": "network-vif-deleted", "tag": "2038175b-b88a-437c-9114-63208c464911"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:01.253157 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-9e1524df-bc1c-4fcf-96b2-1bdf04984662 req-7707227e-d3e8-4095-bea1-024025d8c0a6 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:01.257875 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-9e1524df-bc1c-4fcf-96b2-1bdf04984662 req-7707227e-d3e8-4095-bea1-024025d8c0a6 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:01.268435 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-9e1524df-bc1c-4fcf-96b2-1bdf04984662 req-7707227e-d3e8-4095-bea1-024025d8c0a6 service nova] Creating event network-vif-deleted:2038175b-b88a-437c-9114-63208c464911 for instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 on hv-cinder-84336 May 28 03:42:01.274649 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-9e1524df-bc1c-4fcf-96b2-1bdf04984662 req-7707227e-d3e8-4095-bea1-024025d8c0a6 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042780 May 28 03:42:01.275512 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 427/827] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:42:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:42:01.647328 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7ce3037c-3fc9-41ed-9b12-eec925c29922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:01.647939 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7ce3037c-3fc9-41ed-9b12-eec925c29922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:01.669349 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7ce3037c-3fc9-41ed-9b12-eec925c29922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:01.673901 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7ce3037c-3fc9-41ed-9b12-eec925c29922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:01.673901 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7ce3037c-3fc9-41ed-9b12-eec925c29922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:01.673901 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7ce3037c-3fc9-41ed-9b12-eec925c29922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:01.710024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7ce3037c-3fc9-41ed-9b12-eec925c29922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:01.710745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7ce3037c-3fc9-41ed-9b12-eec925c29922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:01.711898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-7ce3037c-3fc9-41ed-9b12-eec925c29922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lazy-loading 'fault' on Instance uuid 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:42:01.763506 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7ce3037c-3fc9-41ed-9b12-eec925c29922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:01.772163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7ce3037c-3fc9-41ed-9b12-eec925c29922 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1312 microversion: 2.1 time: 0.135644 May 28 03:42:01.773256 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 401/828] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:01 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1312 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:02.134679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-930d6375-1758-4276-a28e-17fc2adc0ed4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:02.135633 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-930d6375-1758-4276-a28e-17fc2adc0ed4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:02.144189 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-930d6375-1758-4276-a28e-17fc2adc0ed4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:02.144827 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-930d6375-1758-4276-a28e-17fc2adc0ed4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:02.148439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-930d6375-1758-4276-a28e-17fc2adc0ed4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:02.149173 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-930d6375-1758-4276-a28e-17fc2adc0ed4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:02.303215 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-930d6375-1758-4276-a28e-17fc2adc0ed4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:02.311977 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-930d6375-1758-4276-a28e-17fc2adc0ed4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:02.313545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-930d6375-1758-4276-a28e-17fc2adc0ed4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:02.315094 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-930d6375-1758-4276-a28e-17fc2adc0ed4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1740 microversion: 2.20 time: 0.185248 May 28 03:42:02.315645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 428/829] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:02 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1740 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:02.793794 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1dacc7bc-2edc-4fa8-8383-7ae355312cde tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:02.795278 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1dacc7bc-2edc-4fa8-8383-7ae355312cde tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:02.801755 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dacc7bc-2edc-4fa8-8383-7ae355312cde tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:02.802205 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dacc7bc-2edc-4fa8-8383-7ae355312cde tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:02.803750 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dacc7bc-2edc-4fa8-8383-7ae355312cde tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:02.804439 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dacc7bc-2edc-4fa8-8383-7ae355312cde tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:02.852820 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dacc7bc-2edc-4fa8-8383-7ae355312cde tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:02.853558 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1dacc7bc-2edc-4fa8-8383-7ae355312cde tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:02.854178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-1dacc7bc-2edc-4fa8-8383-7ae355312cde tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lazy-loading 'fault' on Instance uuid 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:42:02.904801 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1dacc7bc-2edc-4fa8-8383-7ae355312cde tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6709eed72ca46468fd298929ae0f764', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cab8255c6134aebb9c93493a2342cb1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:02.917906 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1dacc7bc-2edc-4fa8-8383-7ae355312cde tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 200 len: 1312 microversion: 2.1 time: 0.130667 May 28 03:42:02.927031 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 402/830] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:02 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 1312 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:03.342330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4aa6de09-175f-462a-8dd0-3064206deb27 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:03.343196 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4aa6de09-175f-462a-8dd0-3064206deb27 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:03.363829 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4aa6de09-175f-462a-8dd0-3064206deb27 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:03.364646 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4aa6de09-175f-462a-8dd0-3064206deb27 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:03.366852 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4aa6de09-175f-462a-8dd0-3064206deb27 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:03.367641 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4aa6de09-175f-462a-8dd0-3064206deb27 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:03.564906 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4aa6de09-175f-462a-8dd0-3064206deb27 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:03.574170 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4aa6de09-175f-462a-8dd0-3064206deb27 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:03.576323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4aa6de09-175f-462a-8dd0-3064206deb27 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:03.577814 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4aa6de09-175f-462a-8dd0-3064206deb27 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1740 microversion: 2.20 time: 0.242597 May 28 03:42:03.578381 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 429/831] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:03 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1740 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:03.583384 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cd7c5947-fcc2-41f4-bd27-963cf579594a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:03.584911 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-cd7c5947-fcc2-41f4-bd27-963cf579594a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:03.591431 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cd7c5947-fcc2-41f4-bd27-963cf579594a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:03.591895 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cd7c5947-fcc2-41f4-bd27-963cf579594a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:03.592677 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cd7c5947-fcc2-41f4-bd27-963cf579594a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:03.593190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cd7c5947-fcc2-41f4-bd27-963cf579594a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:03.626787 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-cd7c5947-fcc2-41f4-bd27-963cf579594a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047250 May 28 03:42:03.627575 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 403/832] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:42:03 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:42:03.637278 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-afd91498-487a-4453-a834-598f79ee912e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:03.646099 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-afd91498-487a-4453-a834-598f79ee912e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:03.646099 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-afd91498-487a-4453-a834-598f79ee912e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:03.646099 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-afd91498-487a-4453-a834-598f79ee912e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:03.646955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-afd91498-487a-4453-a834-598f79ee912e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:03.647193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-afd91498-487a-4453-a834-598f79ee912e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:03.683615 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-afd91498-487a-4453-a834-598f79ee912e tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments/43e258a6-edd7-4013-bbba-387dc62c7053" status: 200 len: 194 microversion: 2.1 time: 0.050104 May 28 03:42:03.684351 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 430/833] 10.222.0.41 () {58 vars in 1412 bytes} [Sat May 28 03:42:03 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments/43e258a6-edd7-4013-bbba-387dc62c7053 => generated 194 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:42:03.939988 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0ef26661-5983-472f-879a-88cc81fc2fc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:03.940653 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0ef26661-5983-472f-879a-88cc81fc2fc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] [instance: 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:03.953213 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0ef26661-5983-472f-879a-88cc81fc2fc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:03.953973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0ef26661-5983-472f-879a-88cc81fc2fc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:03.954932 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0ef26661-5983-472f-879a-88cc81fc2fc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:03.955533 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0ef26661-5983-472f-879a-88cc81fc2fc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:03.972174 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-0ef26661-5983-472f-879a-88cc81fc2fc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] HTTP exception thrown: Instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 could not be found. May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0ef26661-5983-472f-879a-88cc81fc2fc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Request method failure captured: May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 HTTP/1.1 May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZioFSMjchJLpIZqod5q-7lgl8fEY36gE52nylmpmtlBCMGq9sB1-y2xrNX1ntFqjnFW5i92fC2liYIvh0AHl9bCuVQTnAqKQCbialBgg6tn7xuEEa8ZKcsvKvenrdvzcDN3coDLFL7-jztA0Tb3mMeshJqy19Ht6r4OU5cn2edXS_E May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 5cab8255c6134aebb9c93493a2342cb1 May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:42:03.974481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_5cab8255c6134aebb9c93493a2342cb1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 5cab8255c6134aebb9c93493a2342cb1 May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-968064324 May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-ServerRescueNegativeTestJSON-968064324-project May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: b6709eed72ca46468fd298929ae0f764 May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-ServerRescueNegativeTestJSON-968064324-project May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 could not be found. May 28 03:42:03.975908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '3b4a0f56-eb59-4f13-a90a-442b9a2e0af8'} May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 could not be found. May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:03.977295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 could not be found. May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 could not be found. May 28 03:42:03.978551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:03.979668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0ef26661-5983-472f-879a-88cc81fc2fc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] Returning 404 to user: Instance 3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:03.979668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0ef26661-5983-472f-879a-88cc81fc2fc9 tempest-ServerRescueNegativeTestJSON-968064324 tempest-ServerRescueNegativeTestJSON-968064324-project] 10.222.0.41 "GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8" status: 404 len: 111 microversion: 2.1 time: 0.048115 May 28 03:42:03.979668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 404/834] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:03 2022] GET /compute/v2.1/servers/3b4a0f56-eb59-4f13-a90a-442b9a2e0af8 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 28 03:42:04.595853 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c82c2b07-c841-4e3c-8e42-d05923d239f2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:04.596686 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c82c2b07-c841-4e3c-8e42-d05923d239f2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:04.609443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c82c2b07-c841-4e3c-8e42-d05923d239f2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:04.609893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c82c2b07-c841-4e3c-8e42-d05923d239f2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:04.610670 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c82c2b07-c841-4e3c-8e42-d05923d239f2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:04.611103 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c82c2b07-c841-4e3c-8e42-d05923d239f2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:04.911690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c82c2b07-c841-4e3c-8e42-d05923d239f2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:04.919976 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c82c2b07-c841-4e3c-8e42-d05923d239f2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:04.921496 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c82c2b07-c841-4e3c-8e42-d05923d239f2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:04.922659 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c82c2b07-c841-4e3c-8e42-d05923d239f2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1740 microversion: 2.20 time: 0.330762 May 28 03:42:04.923208 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 431/835] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:04 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1740 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:05.600313 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-18e11fab-2022-4f77-91a3-c43c3d838735 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "302e0683-96b4-487c-81c5-dbca4a55cc98"}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:05.603886 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-18e11fab-2022-4f77-91a3-c43c3d838735 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:05.621445 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-18e11fab-2022-4f77-91a3-c43c3d838735 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:05.622731 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-18e11fab-2022-4f77-91a3-c43c3d838735 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:05.624326 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-18e11fab-2022-4f77-91a3-c43c3d838735 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:05.625606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-18e11fab-2022-4f77-91a3-c43c3d838735 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:05.943461 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-70896d8c-bb72-4ca9-99b1-9173186ad273 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:05.948524 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-70896d8c-bb72-4ca9-99b1-9173186ad273 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:05.956815 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70896d8c-bb72-4ca9-99b1-9173186ad273 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:05.957641 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70896d8c-bb72-4ca9-99b1-9173186ad273 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:05.958648 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70896d8c-bb72-4ca9-99b1-9173186ad273 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:05.959872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-70896d8c-bb72-4ca9-99b1-9173186ad273 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:06.333654 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-70896d8c-bb72-4ca9-99b1-9173186ad273 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:06.345922 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-70896d8c-bb72-4ca9-99b1-9173186ad273 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:06.348043 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-70896d8c-bb72-4ca9-99b1-9173186ad273 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:06.349698 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-70896d8c-bb72-4ca9-99b1-9173186ad273 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1740 microversion: 2.20 time: 0.412001 May 28 03:42:06.361051 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 432/836] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:05 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1740 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:06.502951 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-18e11fab-2022-4f77-91a3-c43c3d838735 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "POST /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.907967 May 28 03:42:06.503846 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 405/837] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:42:05 2022] POST /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments => generated 194 bytes in 910 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:42:07.368717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9c0b869a-ec01-4a49-b954-73ee23419d3b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:07.370570 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9c0b869a-ec01-4a49-b954-73ee23419d3b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:07.378344 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9c0b869a-ec01-4a49-b954-73ee23419d3b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:07.379032 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9c0b869a-ec01-4a49-b954-73ee23419d3b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:07.379980 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9c0b869a-ec01-4a49-b954-73ee23419d3b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:07.380425 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9c0b869a-ec01-4a49-b954-73ee23419d3b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:07.520079 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9c0b869a-ec01-4a49-b954-73ee23419d3b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:07.529210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9c0b869a-ec01-4a49-b954-73ee23419d3b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:07.530738 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9c0b869a-ec01-4a49-b954-73ee23419d3b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:07.531909 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9c0b869a-ec01-4a49-b954-73ee23419d3b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1740 microversion: 2.20 time: 0.167656 May 28 03:42:07.532572 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 433/838] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:07 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1740 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:08.550909 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-223b4c4b-d9c0-4507-a437-4bcc5f8105e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:08.551606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-223b4c4b-d9c0-4507-a437-4bcc5f8105e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:08.564038 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-223b4c4b-d9c0-4507-a437-4bcc5f8105e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:08.564518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-223b4c4b-d9c0-4507-a437-4bcc5f8105e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:08.565314 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-223b4c4b-d9c0-4507-a437-4bcc5f8105e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:08.589957 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-223b4c4b-d9c0-4507-a437-4bcc5f8105e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.024s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:08.957725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-223b4c4b-d9c0-4507-a437-4bcc5f8105e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:08.966918 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-223b4c4b-d9c0-4507-a437-4bcc5f8105e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:08.968502 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-223b4c4b-d9c0-4507-a437-4bcc5f8105e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:08.969867 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-223b4c4b-d9c0-4507-a437-4bcc5f8105e2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1740 microversion: 2.20 time: 0.423257 May 28 03:42:08.970425 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 406/839] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:08 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1740 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:09.181446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1339951824", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1597361276", "security_groups": [{"name": "tempest-secgroup-smoke-1971281779"}], "block_device_mapping_v2": [{"uuid": "4b793144-3494-4b23-a6de-e09d47fb7c69", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "af6636fe-9088-4d20-9a1e-fa3df2392948"}]}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:09.299467 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] validate_networks() for [('af6636fe-9088-4d20-9a1e-fa3df2392948', None, None, None, None, None)] {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:42:09.806335 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:42:09.807239 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:42:09.807934 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:42:09.808603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:42:09.809262 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:42:09.820595 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:09.825418 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:09.825418 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:09.825418 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:09.844112 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Getting quotas for project c0fe831f6e954e05b7d448f1d4f43754. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:42:09.848099 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Getting quotas for user 2b515cebf18643c38f1cfc2e3fad1db4 and project c0fe831f6e954e05b7d448f1d4f43754. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:42:09.935691 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:42:09.938093 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='4b793144-3494-4b23-a6de-e09d47fb7c69',volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:42:09.992081 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-311f034e-3b36-4269-9007-63ee06d9c06e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:09.992998 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-311f034e-3b36-4269-9007-63ee06d9c06e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:10.004332 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-311f034e-3b36-4269-9007-63ee06d9c06e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:10.005711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-311f034e-3b36-4269-9007-63ee06d9c06e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:10.007564 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-311f034e-3b36-4269-9007-63ee06d9c06e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:10.008086 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-311f034e-3b36-4269-9007-63ee06d9c06e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:10.179182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-311f034e-3b36-4269-9007-63ee06d9c06e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:10.188291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-311f034e-3b36-4269-9007-63ee06d9c06e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:10.190153 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-311f034e-3b36-4269-9007-63ee06d9c06e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:10.192123 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-311f034e-3b36-4269-9007-63ee06d9c06e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1740 microversion: 2.20 time: 0.204728 May 28 03:42:10.192855 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 407/840] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:09 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1740 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:42:10.546620 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-caed4305-c97c-4f23-88b7-cdb47d2f9ed5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.369111 May 28 03:42:10.549529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 434/841] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:42:09 2022] POST /compute/v2.1/servers => generated 406 bytes in 1372 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:42:10.578191 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-059f27dd-1086-41aa-82e7-dc7a8861733e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:10.579561 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-059f27dd-1086-41aa-82e7-dc7a8861733e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:10.668236 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-059f27dd-1086-41aa-82e7-dc7a8861733e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:10.676117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-059f27dd-1086-41aa-82e7-dc7a8861733e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1095 microversion: 2.1 time: 0.102204 May 28 03:42:10.677201 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 408/842] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:10 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1095 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:11.214036 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0edba780-f1b6-4bf6-a71b-c651ba75d3d1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:11.215090 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0edba780-f1b6-4bf6-a71b-c651ba75d3d1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:11.222268 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0edba780-f1b6-4bf6-a71b-c651ba75d3d1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:11.223219 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0edba780-f1b6-4bf6-a71b-c651ba75d3d1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:11.224654 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0edba780-f1b6-4bf6-a71b-c651ba75d3d1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:11.225222 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0edba780-f1b6-4bf6-a71b-c651ba75d3d1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:11.408256 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0edba780-f1b6-4bf6-a71b-c651ba75d3d1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:11.419031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0edba780-f1b6-4bf6-a71b-c651ba75d3d1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:11.420880 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0edba780-f1b6-4bf6-a71b-c651ba75d3d1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:11.422130 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0edba780-f1b6-4bf6-a71b-c651ba75d3d1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1740 microversion: 2.20 time: 0.212496 May 28 03:42:11.423111 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 435/843] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:11 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1740 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:11.694733 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8b752be5-c141-4787-86a8-d722466c4e9e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:11.696684 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8b752be5-c141-4787-86a8-d722466c4e9e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:11.710151 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8b752be5-c141-4787-86a8-d722466c4e9e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:11.710686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8b752be5-c141-4787-86a8-d722466c4e9e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:11.711543 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8b752be5-c141-4787-86a8-d722466c4e9e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:11.713944 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8b752be5-c141-4787-86a8-d722466c4e9e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:11.792551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8b752be5-c141-4787-86a8-d722466c4e9e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:11.802151 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8b752be5-c141-4787-86a8-d722466c4e9e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1145 microversion: 2.1 time: 0.112797 May 28 03:42:11.804040 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 409/844] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:11 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1145 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:12.447642 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-972f356e-12c3-48f6-a7a7-f5009679ca5e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:12.448277 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-972f356e-12c3-48f6-a7a7-f5009679ca5e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:12.455100 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-972f356e-12c3-48f6-a7a7-f5009679ca5e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:12.455549 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-972f356e-12c3-48f6-a7a7-f5009679ca5e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:12.456315 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-972f356e-12c3-48f6-a7a7-f5009679ca5e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:12.456747 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-972f356e-12c3-48f6-a7a7-f5009679ca5e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:12.603367 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-972f356e-12c3-48f6-a7a7-f5009679ca5e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:12.611119 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-972f356e-12c3-48f6-a7a7-f5009679ca5e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:12.612625 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-972f356e-12c3-48f6-a7a7-f5009679ca5e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:12.614023 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-972f356e-12c3-48f6-a7a7-f5009679ca5e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1740 microversion: 2.20 time: 0.170859 May 28 03:42:12.614577 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 436/845] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:12 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1740 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:12.820496 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9a3e1a38-b95f-4611-8dfd-0b28a054d0ab tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:12.822045 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9a3e1a38-b95f-4611-8dfd-0b28a054d0ab tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:12.828988 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a3e1a38-b95f-4611-8dfd-0b28a054d0ab tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:12.829463 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a3e1a38-b95f-4611-8dfd-0b28a054d0ab tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:12.830345 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a3e1a38-b95f-4611-8dfd-0b28a054d0ab tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:12.830780 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a3e1a38-b95f-4611-8dfd-0b28a054d0ab tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:12.898386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9a3e1a38-b95f-4611-8dfd-0b28a054d0ab tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:12.908236 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9a3e1a38-b95f-4611-8dfd-0b28a054d0ab tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1193 microversion: 2.1 time: 0.092184 May 28 03:42:12.909645 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 410/846] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:12 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1193 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:13.632921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8b83f5e7-1d7d-49d4-beeb-ca1725551826 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:13.633575 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8b83f5e7-1d7d-49d4-beeb-ca1725551826 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:13.640141 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8b83f5e7-1d7d-49d4-beeb-ca1725551826 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:13.640590 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8b83f5e7-1d7d-49d4-beeb-ca1725551826 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:13.641584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8b83f5e7-1d7d-49d4-beeb-ca1725551826 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:13.642029 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8b83f5e7-1d7d-49d4-beeb-ca1725551826 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:13.777562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8b83f5e7-1d7d-49d4-beeb-ca1725551826 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:13.789283 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8b83f5e7-1d7d-49d4-beeb-ca1725551826 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:13.789283 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8b83f5e7-1d7d-49d4-beeb-ca1725551826 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:13.789283 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8b83f5e7-1d7d-49d4-beeb-ca1725551826 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1740 microversion: 2.20 time: 0.160042 May 28 03:42:13.789680 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 437/847] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:13 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1740 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:13.925817 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-51fcfc89-c3bf-47d3-90c8-4b874441ba35 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:13.926424 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-51fcfc89-c3bf-47d3-90c8-4b874441ba35 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:13.933576 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-51fcfc89-c3bf-47d3-90c8-4b874441ba35 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:13.934048 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-51fcfc89-c3bf-47d3-90c8-4b874441ba35 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:13.934807 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-51fcfc89-c3bf-47d3-90c8-4b874441ba35 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:13.935263 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-51fcfc89-c3bf-47d3-90c8-4b874441ba35 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:14.031390 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-51fcfc89-c3bf-47d3-90c8-4b874441ba35 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:14.040322 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-51fcfc89-c3bf-47d3-90c8-4b874441ba35 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1211 microversion: 2.1 time: 0.118335 May 28 03:42:14.041233 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 411/848] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:13 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1211 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:14.807738 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bee40a16-bd19-4772-be93-140ff9e6bb5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:14.808617 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bee40a16-bd19-4772-be93-140ff9e6bb5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:14.815645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bee40a16-bd19-4772-be93-140ff9e6bb5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:14.816175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bee40a16-bd19-4772-be93-140ff9e6bb5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:14.817152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bee40a16-bd19-4772-be93-140ff9e6bb5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:14.817613 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bee40a16-bd19-4772-be93-140ff9e6bb5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:14.989483 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bee40a16-bd19-4772-be93-140ff9e6bb5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:15.000394 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bee40a16-bd19-4772-be93-140ff9e6bb5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:15.002097 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bee40a16-bd19-4772-be93-140ff9e6bb5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:15.003786 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bee40a16-bd19-4772-be93-140ff9e6bb5c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1722 microversion: 2.20 time: 0.200106 May 28 03:42:15.004580 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 438/849] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:14 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1722 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:42:15.060264 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9a7de277-67c8-465e-a062-84e8f84c527e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:15.061521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9a7de277-67c8-465e-a062-84e8f84c527e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:15.068748 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a7de277-67c8-465e-a062-84e8f84c527e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:15.069326 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a7de277-67c8-465e-a062-84e8f84c527e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:15.070102 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a7de277-67c8-465e-a062-84e8f84c527e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:15.070620 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a7de277-67c8-465e-a062-84e8f84c527e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:15.285712 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a67aa7a8-255f-4ef8-b6ed-e5bff6096ac2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:15.287446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a67aa7a8-255f-4ef8-b6ed-e5bff6096ac2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:15.293502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a67aa7a8-255f-4ef8-b6ed-e5bff6096ac2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:15.293951 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a67aa7a8-255f-4ef8-b6ed-e5bff6096ac2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:15.294732 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a67aa7a8-255f-4ef8-b6ed-e5bff6096ac2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:15.295169 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a67aa7a8-255f-4ef8-b6ed-e5bff6096ac2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:15.299778 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:15.327529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a67aa7a8-255f-4ef8-b6ed-e5bff6096ac2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.045585 May 28 03:42:15.333274 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 439/850] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:42:15 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments => generated 371 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:42:15.466528 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2fd0fb14-7d76-4be1-b182-95b0f46154cf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:15.470072 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9a7de277-67c8-465e-a062-84e8f84c527e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:15.474223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2fd0fb14-7d76-4be1-b182-95b0f46154cf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:15.481336 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9a7de277-67c8-465e-a062-84e8f84c527e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1279 microversion: 2.1 time: 0.424111 May 28 03:42:15.482120 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2fd0fb14-7d76-4be1-b182-95b0f46154cf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:15.483176 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 412/851] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:15 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1279 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:15.491186 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2fd0fb14-7d76-4be1-b182-95b0f46154cf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:15.492106 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2fd0fb14-7d76-4be1-b182-95b0f46154cf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:15.492605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2fd0fb14-7d76-4be1-b182-95b0f46154cf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:15.533194 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2fd0fb14-7d76-4be1-b182-95b0f46154cf tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments/43e258a6-edd7-4013-bbba-387dc62c7053" status: 200 len: 194 microversion: 2.1 time: 0.071176 May 28 03:42:15.534010 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 440/852] 10.222.0.41 () {58 vars in 1412 bytes} [Sat May 28 03:42:15 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments/43e258a6-edd7-4013-bbba-387dc62c7053 => generated 194 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:42:15.545077 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a960fb77-de7f-4be7-a13f-e3b1119aa259 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:15.545682 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a960fb77-de7f-4be7-a13f-e3b1119aa259 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:15.552437 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a960fb77-de7f-4be7-a13f-e3b1119aa259 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:15.553310 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a960fb77-de7f-4be7-a13f-e3b1119aa259 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:15.554140 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a960fb77-de7f-4be7-a13f-e3b1119aa259 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:15.554575 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a960fb77-de7f-4be7-a13f-e3b1119aa259 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:15.641397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:15.645250 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-a960fb77-de7f-4be7-a13f-e3b1119aa259 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lazy-loading 'services' on Instance uuid 43942790-5855-4ed1-9ce4-8f02a3092edd {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:42:15.901464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a960fb77-de7f-4be7-a13f-e3b1119aa259 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "DELETE /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments/43e258a6-edd7-4013-bbba-387dc62c7053" status: 202 len: 0 microversion: 2.1 time: 0.360625 May 28 03:42:15.907987 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 413/853] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:42:15 2022] DELETE /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments/43e258a6-edd7-4013-bbba-387dc62c7053 => generated 0 bytes in 362 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:42:16.027808 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dd4c5f95-7d98-4bd9-b575-86e82bd2694c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:16.028421 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dd4c5f95-7d98-4bd9-b575-86e82bd2694c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:16.035514 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd4c5f95-7d98-4bd9-b575-86e82bd2694c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:16.035916 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd4c5f95-7d98-4bd9-b575-86e82bd2694c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:16.036694 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd4c5f95-7d98-4bd9-b575-86e82bd2694c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:16.037850 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dd4c5f95-7d98-4bd9-b575-86e82bd2694c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:16.130565 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-6eccd9f9-6a46-4549-a46c-e40a9db76123 req-16a9de76-8610-47c8-882a-c8f7405b89ff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b53491f6-a319-4950-ac9e-a9b0bd7a45e2", "tag": "ce3985bf-d85c-4d8a-be10-2df6ad569608"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:16.146704 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-6eccd9f9-6a46-4549-a46c-e40a9db76123 req-16a9de76-8610-47c8-882a-c8f7405b89ff service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:16.147137 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-6eccd9f9-6a46-4549-a46c-e40a9db76123 req-16a9de76-8610-47c8-882a-c8f7405b89ff service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:16.164679 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-6eccd9f9-6a46-4549-a46c-e40a9db76123 req-16a9de76-8610-47c8-882a-c8f7405b89ff service nova] Creating event network-changed:ce3985bf-d85c-4d8a-be10-2df6ad569608 for instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 on hv-cinder-84336 May 28 03:42:16.173567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-6eccd9f9-6a46-4549-a46c-e40a9db76123 req-16a9de76-8610-47c8-882a-c8f7405b89ff service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046727 May 28 03:42:16.177346 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 414/854] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:42:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:42:16.190620 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dd4c5f95-7d98-4bd9-b575-86e82bd2694c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:16.199086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dd4c5f95-7d98-4bd9-b575-86e82bd2694c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:16.201414 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dd4c5f95-7d98-4bd9-b575-86e82bd2694c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:16.206935 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-dd4c5f95-7d98-4bd9-b575-86e82bd2694c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1722 microversion: 2.20 time: 0.178755 May 28 03:42:16.206935 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 441/855] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:16 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1722 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:16.499734 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-efe27887-f9ec-4149-b54c-e06067404b0b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:16.500435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-efe27887-f9ec-4149-b54c-e06067404b0b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:16.507086 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-efe27887-f9ec-4149-b54c-e06067404b0b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:16.507537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-efe27887-f9ec-4149-b54c-e06067404b0b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:16.508306 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-efe27887-f9ec-4149-b54c-e06067404b0b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:16.508742 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-efe27887-f9ec-4149-b54c-e06067404b0b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:16.644694 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-efe27887-f9ec-4149-b54c-e06067404b0b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:16.652996 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-efe27887-f9ec-4149-b54c-e06067404b0b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1279 microversion: 2.1 time: 0.157797 May 28 03:42:16.653598 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 415/856] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:16 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1279 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:17.221621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ea885403-af03-47e5-9e77-2c057d1f8d4c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:17.223363 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ea885403-af03-47e5-9e77-2c057d1f8d4c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:17.231819 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ea885403-af03-47e5-9e77-2c057d1f8d4c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:17.232502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ea885403-af03-47e5-9e77-2c057d1f8d4c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:17.233516 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ea885403-af03-47e5-9e77-2c057d1f8d4c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:17.234167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ea885403-af03-47e5-9e77-2c057d1f8d4c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:17.383672 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ea885403-af03-47e5-9e77-2c057d1f8d4c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:17.391761 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ea885403-af03-47e5-9e77-2c057d1f8d4c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:17.393396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ea885403-af03-47e5-9e77-2c057d1f8d4c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:17.394546 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ea885403-af03-47e5-9e77-2c057d1f8d4c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1722 microversion: 2.20 time: 0.177204 May 28 03:42:17.395097 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 442/857] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:17 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1722 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:17.671576 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-fe866c18-e7dd-443a-98d5-a44a52e43654 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:17.672351 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-fe866c18-e7dd-443a-98d5-a44a52e43654 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:17.679820 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fe866c18-e7dd-443a-98d5-a44a52e43654 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:17.680260 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fe866c18-e7dd-443a-98d5-a44a52e43654 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:17.681032 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fe866c18-e7dd-443a-98d5-a44a52e43654 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:17.681483 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fe866c18-e7dd-443a-98d5-a44a52e43654 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:17.817967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-fe866c18-e7dd-443a-98d5-a44a52e43654 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:17.826367 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-fe866c18-e7dd-443a-98d5-a44a52e43654 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.158543 May 28 03:42:17.829140 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 416/858] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:17 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:18.414091 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0eb6b8d1-44a2-4a8f-8fa5-55b5941e3d67 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:18.421341 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0eb6b8d1-44a2-4a8f-8fa5-55b5941e3d67 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:18.421866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0eb6b8d1-44a2-4a8f-8fa5-55b5941e3d67 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:18.422319 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0eb6b8d1-44a2-4a8f-8fa5-55b5941e3d67 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:18.423148 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0eb6b8d1-44a2-4a8f-8fa5-55b5941e3d67 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:18.423616 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0eb6b8d1-44a2-4a8f-8fa5-55b5941e3d67 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:18.554847 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0eb6b8d1-44a2-4a8f-8fa5-55b5941e3d67 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:18.562984 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0eb6b8d1-44a2-4a8f-8fa5-55b5941e3d67 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:18.564490 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0eb6b8d1-44a2-4a8f-8fa5-55b5941e3d67 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:18.565689 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0eb6b8d1-44a2-4a8f-8fa5-55b5941e3d67 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1665 microversion: 2.20 time: 0.157728 May 28 03:42:18.566525 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 443/859] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:18 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1665 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:18.584875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5dd57b5c-98e1-4c8a-8bcf-889d709fe0cf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d41db59d-1c68-482e-898c-01dd8b1150b4"}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:18.585967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5dd57b5c-98e1-4c8a-8bcf-889d709fe0cf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:18.592193 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5dd57b5c-98e1-4c8a-8bcf-889d709fe0cf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:18.592644 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5dd57b5c-98e1-4c8a-8bcf-889d709fe0cf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:18.593486 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5dd57b5c-98e1-4c8a-8bcf-889d709fe0cf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:18.593925 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5dd57b5c-98e1-4c8a-8bcf-889d709fe0cf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:18.666837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-ab81d051-c5be-4e4b-b8bb-2f50915b753f req-2d305a7a-37c9-43e3-b785-6e4afa0414ee service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6be2aeaa-8d90-4e51-87d2-d02a8566ac1a", "name": "network-vif-unplugged", "status": "completed", "tag": "73f87b73-9d7c-43a7-9bd2-dc7fb5450112"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:18.678417 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-ab81d051-c5be-4e4b-b8bb-2f50915b753f req-2d305a7a-37c9-43e3-b785-6e4afa0414ee service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:18.678972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-ab81d051-c5be-4e4b-b8bb-2f50915b753f req-2d305a7a-37c9-43e3-b785-6e4afa0414ee service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:18.691446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.compute.server_external_events [req-ab81d051-c5be-4e4b-b8bb-2f50915b753f req-2d305a7a-37c9-43e3-b785-6e4afa0414ee service nova] Unable to find a host for instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a. Dropping event network-vif-unplugged {{(pid=124604) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} May 28 03:42:18.692252 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-ab81d051-c5be-4e4b-b8bb-2f50915b753f req-2d305a7a-37c9-43e3-b785-6e4afa0414ee service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.029176 May 28 03:42:18.693368 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 444/860] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:42:18 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 31 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) May 28 03:42:18.843861 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dde5789c-db70-442d-a7ef-400a4bb2a16e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:18.844481 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dde5789c-db70-442d-a7ef-400a4bb2a16e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:18.851194 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dde5789c-db70-442d-a7ef-400a4bb2a16e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:18.851772 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dde5789c-db70-442d-a7ef-400a4bb2a16e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:18.852708 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dde5789c-db70-442d-a7ef-400a4bb2a16e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:18.853216 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-dde5789c-db70-442d-a7ef-400a4bb2a16e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:18.991266 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dde5789c-db70-442d-a7ef-400a4bb2a16e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:18.999981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-dde5789c-db70-442d-a7ef-400a4bb2a16e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.160191 May 28 03:42:19.000544 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 445/861] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:18 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:42:19.157517 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d8193b03-aa02-4df1-abae-0db7eaf4e643 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:19.158292 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d8193b03-aa02-4df1-abae-0db7eaf4e643 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:19.161863 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=89,nova_api:DELETE=1,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:19.165764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d8193b03-aa02-4df1-abae-0db7eaf4e643 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:19.166443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d8193b03-aa02-4df1-abae-0db7eaf4e643 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:19.167506 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d8193b03-aa02-4df1-abae-0db7eaf4e643 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:19.167837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d8193b03-aa02-4df1-abae-0db7eaf4e643 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:19.198972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d8193b03-aa02-4df1-abae-0db7eaf4e643 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments/302e0683-96b4-487c-81c5-dbca4a55cc98" status: 200 len: 194 microversion: 2.1 time: 0.045560 May 28 03:42:19.199764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 446/862] 10.222.0.41 () {58 vars in 1412 bytes} [Sat May 28 03:42:19 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments/302e0683-96b4-487c-81c5-dbca4a55cc98 => generated 194 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:42:19.210321 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-07eaf2d1-d76b-4ab3-897e-3f8bf1268e4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:19.211043 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-07eaf2d1-d76b-4ab3-897e-3f8bf1268e4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:19.216890 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07eaf2d1-d76b-4ab3-897e-3f8bf1268e4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:19.217372 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07eaf2d1-d76b-4ab3-897e-3f8bf1268e4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:19.218322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07eaf2d1-d76b-4ab3-897e-3f8bf1268e4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:19.218754 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07eaf2d1-d76b-4ab3-897e-3f8bf1268e4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:19.309999 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-07eaf2d1-d76b-4ab3-897e-3f8bf1268e4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lazy-loading 'services' on Instance uuid 43942790-5855-4ed1-9ce4-8f02a3092edd {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:42:19.558228 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-07eaf2d1-d76b-4ab3-897e-3f8bf1268e4a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "DELETE /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments/302e0683-96b4-487c-81c5-dbca4a55cc98" status: 202 len: 0 microversion: 2.1 time: 0.351526 May 28 03:42:19.558936 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 447/863] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:42:19 2022] DELETE /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments/302e0683-96b4-487c-81c5-dbca4a55cc98 => generated 0 bytes in 353 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 28 03:42:19.744087 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5dd57b5c-98e1-4c8a-8bcf-889d709fe0cf tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "POST /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.162722 May 28 03:42:19.744087 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 417/864] 10.222.0.41 () {62 vars in 1363 bytes} [Sat May 28 03:42:18 2022] POST /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/os-volume_attachments => generated 188 bytes in 1164 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 28 03:42:19.862665 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=2 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:19.863432 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=6 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:19.868220 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-850d39db-1f17-439f-8884-6b5818d18f4d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:19.869129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-850d39db-1f17-439f-8884-6b5818d18f4d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:19.875788 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-850d39db-1f17-439f-8884-6b5818d18f4d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:19.876096 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-850d39db-1f17-439f-8884-6b5818d18f4d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:19.876870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-850d39db-1f17-439f-8884-6b5818d18f4d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:19.877343 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-850d39db-1f17-439f-8884-6b5818d18f4d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.017200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6ad746fa-e4b0-4581-b644-b1fd399cf28d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:20.017823 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6ad746fa-e4b0-4581-b644-b1fd399cf28d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:20.025006 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6ad746fa-e4b0-4581-b644-b1fd399cf28d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.025492 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6ad746fa-e4b0-4581-b644-b1fd399cf28d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.026263 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6ad746fa-e4b0-4581-b644-b1fd399cf28d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.026705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6ad746fa-e4b0-4581-b644-b1fd399cf28d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.173858 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6ad746fa-e4b0-4581-b644-b1fd399cf28d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:20.184088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6ad746fa-e4b0-4581-b644-b1fd399cf28d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.168872 May 28 03:42:20.184088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 418/865] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:20 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:20.443510 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-850d39db-1f17-439f-8884-6b5818d18f4d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "DELETE /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/os-volume_attachments/d41db59d-1c68-482e-898c-01dd8b1150b4" status: 202 len: 0 microversion: 2.20 time: 0.581599 May 28 03:42:20.444404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 448/866] 10.222.0.41 () {60 vars in 1456 bytes} [Sat May 28 03:42:19 2022] DELETE /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/os-volume_attachments/d41db59d-1c68-482e-898c-01dd8b1150b4 => generated 0 bytes in 583 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 28 03:42:20.616973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9e459578-f303-43c9-a0d6-fe714caa015b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:20.617594 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9e459578-f303-43c9-a0d6-fe714caa015b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:20.623826 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9e459578-f303-43c9-a0d6-fe714caa015b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.624269 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9e459578-f303-43c9-a0d6-fe714caa015b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.625019 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9e459578-f303-43c9-a0d6-fe714caa015b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.625470 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9e459578-f303-43c9-a0d6-fe714caa015b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.675333 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:20.735722 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-570f0e2c-d460-41f0-bf1c-9d3b93053988 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:20.736708 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-570f0e2c-d460-41f0-bf1c-9d3b93053988 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:20.743298 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-570f0e2c-d460-41f0-bf1c-9d3b93053988 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.744045 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-570f0e2c-d460-41f0-bf1c-9d3b93053988 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.744822 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-570f0e2c-d460-41f0-bf1c-9d3b93053988 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.745299 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-570f0e2c-d460-41f0-bf1c-9d3b93053988 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.773327 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-570f0e2c-d460-41f0-bf1c-9d3b93053988 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041582 May 28 03:42:20.773885 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 449/867] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:42:20 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) May 28 03:42:20.882509 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4a3ac2c6-81e3-43ce-a4ed-189773c8e458 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:20.883438 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4a3ac2c6-81e3-43ce-a4ed-189773c8e458 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:20.890065 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a3ac2c6-81e3-43ce-a4ed-189773c8e458 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.890524 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a3ac2c6-81e3-43ce-a4ed-189773c8e458 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.891305 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a3ac2c6-81e3-43ce-a4ed-189773c8e458 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.891747 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a3ac2c6-81e3-43ce-a4ed-189773c8e458 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.897797 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9e459578-f303-43c9-a0d6-fe714caa015b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "POST /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/action" status: 202 len: 0 microversion: 2.20 time: 0.285275 May 28 03:42:20.906391 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 419/868] 10.222.0.41 () {62 vars in 1318 bytes} [Sat May 28 03:42:20 2022] POST /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/action => generated 0 bytes in 287 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 28 03:42:20.918973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0c8ef827-0f2a-4eda-87cc-26107ac514e1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:20.919529 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0c8ef827-0f2a-4eda-87cc-26107ac514e1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:20.924745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4a3ac2c6-81e3-43ce-a4ed-189773c8e458 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046143 May 28 03:42:20.925552 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 450/869] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:42:20 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) May 28 03:42:20.927563 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c8ef827-0f2a-4eda-87cc-26107ac514e1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.928007 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c8ef827-0f2a-4eda-87cc-26107ac514e1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.928921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c8ef827-0f2a-4eda-87cc-26107ac514e1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.929438 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c8ef827-0f2a-4eda-87cc-26107ac514e1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.989490 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a778ee97-b7b1-4a53-a19a-97609a557246 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:20.990104 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a778ee97-b7b1-4a53-a19a-97609a557246 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:20.996418 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a778ee97-b7b1-4a53-a19a-97609a557246 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.996863 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a778ee97-b7b1-4a53-a19a-97609a557246 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:20.997672 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a778ee97-b7b1-4a53-a19a-97609a557246 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:20.998101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a778ee97-b7b1-4a53-a19a-97609a557246 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:21.028294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a778ee97-b7b1-4a53-a19a-97609a557246 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Going to try to terminate instance {{(pid=124604) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:42:21.077633 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0c8ef827-0f2a-4eda-87cc-26107ac514e1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:21.086327 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0c8ef827-0f2a-4eda-87cc-26107ac514e1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:21.090122 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0c8ef827-0f2a-4eda-87cc-26107ac514e1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:21.090122 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0c8ef827-0f2a-4eda-87cc-26107ac514e1 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1673 microversion: 2.20 time: 0.179676 May 28 03:42:21.090122 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 420/870] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:20 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1673 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:21.157831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=3,nova_cell1:UPDATE=5 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:21.199382 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2b83e926-ef0e-4598-a9af-151a48e2ef9b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:21.199924 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2b83e926-ef0e-4598-a9af-151a48e2ef9b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:21.206071 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2b83e926-ef0e-4598-a9af-151a48e2ef9b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:21.206808 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2b83e926-ef0e-4598-a9af-151a48e2ef9b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:21.207342 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2b83e926-ef0e-4598-a9af-151a48e2ef9b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:21.207789 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2b83e926-ef0e-4598-a9af-151a48e2ef9b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:21.364431 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a778ee97-b7b1-4a53-a19a-97609a557246 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "DELETE /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 204 len: 0 microversion: 2.1 time: 0.378554 May 28 03:42:21.365243 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 451/871] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:42:20 2022] DELETE /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 0 bytes in 380 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 28 03:42:21.374204 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c4923c07-e7e6-4559-959c-d67c7c4bdc09 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:21.374939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c4923c07-e7e6-4559-959c-d67c7c4bdc09 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:21.381565 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4923c07-e7e6-4559-959c-d67c7c4bdc09 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:21.382016 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4923c07-e7e6-4559-959c-d67c7c4bdc09 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:21.382799 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4923c07-e7e6-4559-959c-d67c7c4bdc09 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:21.383232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4923c07-e7e6-4559-959c-d67c7c4bdc09 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:21.385048 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2b83e926-ef0e-4598-a9af-151a48e2ef9b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:21.401695 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2b83e926-ef0e-4598-a9af-151a48e2ef9b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.205381 May 28 03:42:21.402602 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 421/872] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:21 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:42:21.520591 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c4923c07-e7e6-4559-959c-d67c7c4bdc09 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:21.529217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c4923c07-e7e6-4559-959c-d67c7c4bdc09 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1672 microversion: 2.1 time: 0.159348 May 28 03:42:21.529794 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 452/873] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:21 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1672 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:42:22.107252 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-baa4599e-084d-4642-acbe-39e9ad317990 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:22.107926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-baa4599e-084d-4642-acbe-39e9ad317990 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:22.110376 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:22.115336 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-baa4599e-084d-4642-acbe-39e9ad317990 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:22.115784 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-baa4599e-084d-4642-acbe-39e9ad317990 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:22.117289 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-baa4599e-084d-4642-acbe-39e9ad317990 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:22.117728 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-baa4599e-084d-4642-acbe-39e9ad317990 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:22.252240 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-baa4599e-084d-4642-acbe-39e9ad317990 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:22.260757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-baa4599e-084d-4642-acbe-39e9ad317990 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:22.262329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-baa4599e-084d-4642-acbe-39e9ad317990 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:22.263470 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-baa4599e-084d-4642-acbe-39e9ad317990 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1673 microversion: 2.20 time: 0.160181 May 28 03:42:22.264024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 422/874] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:22 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1673 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:42:22.420200 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b55978e5-ddd0-4922-854a-fba9c49ee754 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:22.420890 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b55978e5-ddd0-4922-854a-fba9c49ee754 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:22.427898 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b55978e5-ddd0-4922-854a-fba9c49ee754 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:22.428342 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b55978e5-ddd0-4922-854a-fba9c49ee754 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:22.429234 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b55978e5-ddd0-4922-854a-fba9c49ee754 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:22.429669 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b55978e5-ddd0-4922-854a-fba9c49ee754 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:22.546565 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-adc85f3c-50f9-45be-9949-7a95f766f2af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:22.547422 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-adc85f3c-50f9-45be-9949-7a95f766f2af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:22.554131 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-adc85f3c-50f9-45be-9949-7a95f766f2af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:22.554765 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-adc85f3c-50f9-45be-9949-7a95f766f2af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:22.558339 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-adc85f3c-50f9-45be-9949-7a95f766f2af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:22.558339 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-adc85f3c-50f9-45be-9949-7a95f766f2af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:22.565180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b55978e5-ddd0-4922-854a-fba9c49ee754 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:22.575707 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b55978e5-ddd0-4922-854a-fba9c49ee754 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.159602 May 28 03:42:22.583310 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 453/875] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:22 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:22.688345 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-adc85f3c-50f9-45be-9949-7a95f766f2af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:22.696460 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-adc85f3c-50f9-45be-9949-7a95f766f2af tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1672 microversion: 2.1 time: 0.153685 May 28 03:42:22.697037 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 423/876] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:22 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1672 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:23.282374 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-34d0c9da-5a4f-4f77-ae73-9c39fe6b721c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:23.283083 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-34d0c9da-5a4f-4f77-ae73-9c39fe6b721c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:23.289701 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-34d0c9da-5a4f-4f77-ae73-9c39fe6b721c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:23.290156 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-34d0c9da-5a4f-4f77-ae73-9c39fe6b721c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:23.290920 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-34d0c9da-5a4f-4f77-ae73-9c39fe6b721c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:23.291352 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-34d0c9da-5a4f-4f77-ae73-9c39fe6b721c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:23.441058 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-34d0c9da-5a4f-4f77-ae73-9c39fe6b721c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:23.449364 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-34d0c9da-5a4f-4f77-ae73-9c39fe6b721c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:23.450875 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-34d0c9da-5a4f-4f77-ae73-9c39fe6b721c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:23.455862 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-34d0c9da-5a4f-4f77-ae73-9c39fe6b721c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1675 microversion: 2.20 time: 0.173655 May 28 03:42:23.455862 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 454/877] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:23 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1675 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:23.594494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4d4a99d7-2634-4f2b-a697-b6f53fc96d79 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:23.595919 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4d4a99d7-2634-4f2b-a697-b6f53fc96d79 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:23.603461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d4a99d7-2634-4f2b-a697-b6f53fc96d79 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:23.603905 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d4a99d7-2634-4f2b-a697-b6f53fc96d79 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:23.604665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d4a99d7-2634-4f2b-a697-b6f53fc96d79 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:23.605090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d4a99d7-2634-4f2b-a697-b6f53fc96d79 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:23.713960 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d1650374-3320-4ca2-bd3f-ae5cee741362 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:23.714626 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d1650374-3320-4ca2-bd3f-ae5cee741362 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:23.720645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d1650374-3320-4ca2-bd3f-ae5cee741362 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:23.721363 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d1650374-3320-4ca2-bd3f-ae5cee741362 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:23.722137 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d1650374-3320-4ca2-bd3f-ae5cee741362 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:23.722755 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d1650374-3320-4ca2-bd3f-ae5cee741362 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:23.765229 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4d4a99d7-2634-4f2b-a697-b6f53fc96d79 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:23.772918 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4d4a99d7-2634-4f2b-a697-b6f53fc96d79 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.182773 May 28 03:42:23.773719 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 424/878] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:23 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:23.864296 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d1650374-3320-4ca2-bd3f-ae5cee741362 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:23.873315 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d1650374-3320-4ca2-bd3f-ae5cee741362 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1672 microversion: 2.1 time: 0.163390 May 28 03:42:23.873905 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 455/879] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:23 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1672 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:42:24.124557 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-bd77f3a3-2eef-4b83-b858-62e37d1450e6 req-2a6c2b75-40f5-43df-bafc-aa705717a90c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43942790-5855-4ed1-9ce4-8f02a3092edd", "name": "network-vif-deleted", "tag": "741e41e9-4e3d-40e3-80a8-4a50ecab9f1e"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:24.132971 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-bd77f3a3-2eef-4b83-b858-62e37d1450e6 req-2a6c2b75-40f5-43df-bafc-aa705717a90c service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:24.133570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-bd77f3a3-2eef-4b83-b858-62e37d1450e6 req-2a6c2b75-40f5-43df-bafc-aa705717a90c service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:24.145756 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-bd77f3a3-2eef-4b83-b858-62e37d1450e6 req-2a6c2b75-40f5-43df-bafc-aa705717a90c service nova] Creating event network-vif-deleted:741e41e9-4e3d-40e3-80a8-4a50ecab9f1e for instance 43942790-5855-4ed1-9ce4-8f02a3092edd on hv-cinder-84336 May 28 03:42:24.160978 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-bd77f3a3-2eef-4b83-b858-62e37d1450e6 req-2a6c2b75-40f5-43df-bafc-aa705717a90c service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031749 May 28 03:42:24.160978 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 425/880] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:42:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:42:24.470979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1876e0bc-4bc2-4ec5-adf8-2e095813a50f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:24.471698 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1876e0bc-4bc2-4ec5-adf8-2e095813a50f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:24.479359 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1876e0bc-4bc2-4ec5-adf8-2e095813a50f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:24.480079 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1876e0bc-4bc2-4ec5-adf8-2e095813a50f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:24.481039 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1876e0bc-4bc2-4ec5-adf8-2e095813a50f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:24.481824 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1876e0bc-4bc2-4ec5-adf8-2e095813a50f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:24.654003 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1876e0bc-4bc2-4ec5-adf8-2e095813a50f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:24.667051 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1876e0bc-4bc2-4ec5-adf8-2e095813a50f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:24.668901 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1876e0bc-4bc2-4ec5-adf8-2e095813a50f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:24.670272 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1876e0bc-4bc2-4ec5-adf8-2e095813a50f tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.203200 May 28 03:42:24.670936 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 456/881] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:24 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:24.792865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4cef3cf8-b7f2-4fe1-93c0-69ca4b5b5315 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:24.794244 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4cef3cf8-b7f2-4fe1-93c0-69ca4b5b5315 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:24.802718 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4cef3cf8-b7f2-4fe1-93c0-69ca4b5b5315 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:24.802718 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4cef3cf8-b7f2-4fe1-93c0-69ca4b5b5315 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:24.802718 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4cef3cf8-b7f2-4fe1-93c0-69ca4b5b5315 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:24.802995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4cef3cf8-b7f2-4fe1-93c0-69ca4b5b5315 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:24.890729 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6586a87e-4c62-437f-84a0-d30d61bf43b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:24.894837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-6586a87e-4c62-437f-84a0-d30d61bf43b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:24.899677 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6586a87e-4c62-437f-84a0-d30d61bf43b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:24.900251 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6586a87e-4c62-437f-84a0-d30d61bf43b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:24.901005 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6586a87e-4c62-437f-84a0-d30d61bf43b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:24.901474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6586a87e-4c62-437f-84a0-d30d61bf43b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:24.954409 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6586a87e-4c62-437f-84a0-d30d61bf43b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:24.954874 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6586a87e-4c62-437f-84a0-d30d61bf43b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:24.955339 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-6586a87e-4c62-437f-84a0-d30d61bf43b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lazy-loading 'fault' on Instance uuid 43942790-5855-4ed1-9ce4-8f02a3092edd {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:42:25.004324 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6586a87e-4c62-437f-84a0-d30d61bf43b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc6a3335562448d1934fa18e8f51b119', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:25.017737 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6586a87e-4c62-437f-84a0-d30d61bf43b2 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 200 len: 1326 microversion: 2.1 time: 0.130628 May 28 03:42:25.033433 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 457/882] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:24 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 1326 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:25.317016 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4cef3cf8-b7f2-4fe1-93c0-69ca4b5b5315 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:25.325430 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4cef3cf8-b7f2-4fe1-93c0-69ca4b5b5315 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.537834 May 28 03:42:25.326006 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 426/883] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:24 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:25.692266 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9689d3af-25f2-4815-9559-89520ad894bd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:25.693457 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9689d3af-25f2-4815-9559-89520ad894bd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:25.701022 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9689d3af-25f2-4815-9559-89520ad894bd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:25.701508 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9689d3af-25f2-4815-9559-89520ad894bd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:25.702298 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9689d3af-25f2-4815-9559-89520ad894bd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:25.702744 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9689d3af-25f2-4815-9559-89520ad894bd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:25.858750 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9689d3af-25f2-4815-9559-89520ad894bd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:25.868172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9689d3af-25f2-4815-9559-89520ad894bd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:25.869977 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9689d3af-25f2-4815-9559-89520ad894bd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:25.871147 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9689d3af-25f2-4815-9559-89520ad894bd tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.187150 May 28 03:42:25.871713 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 458/884] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:25 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:26.035812 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cdd82449-f1dd-4d94-b4de-7140d3823c3a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:26.036427 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-cdd82449-f1dd-4d94-b4de-7140d3823c3a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:26.043654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cdd82449-f1dd-4d94-b4de-7140d3823c3a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:26.044103 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cdd82449-f1dd-4d94-b4de-7140d3823c3a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:26.044867 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cdd82449-f1dd-4d94-b4de-7140d3823c3a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:26.045328 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cdd82449-f1dd-4d94-b4de-7140d3823c3a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:26.058506 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-cdd82449-f1dd-4d94-b4de-7140d3823c3a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] HTTP exception thrown: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cdd82449-f1dd-4d94-b4de-7140d3823c3a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Request method failure captured: May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd HTTP/1.1 May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZiiQypFQQ1bdISGQo9C2YguYEEU-tBqQGVthYz_izW7K5gHqUD5gE5Vtsn9MQmMxa3fO2B2uT4EMCIaBoWDOcTCa_6QL577b2i5ZTpp-2FFBn6eD7fm9RJIlmaKR4ZpoEyCHy7RVgamveJZw6aiAUZ4N9PvH67OUDmLYqMGBctoWpU May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:42:26.060164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: dc6a3335562448d1934fa18e8f51b119 May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:26.061848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '43942790-5855-4ed1-9ce4-8f02a3092edd'} May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:26.062845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:26.064527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:26.066108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cdd82449-f1dd-4d94-b4de-7140d3823c3a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Returning 404 to user: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:26.066108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-cdd82449-f1dd-4d94-b4de-7140d3823c3a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 404 len: 111 microversion: 2.1 time: 0.029705 May 28 03:42:26.066108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 427/885] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:26 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:42:26.070325 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-028a6194-e7e4-4b43-8cb2-cb10fe7f601a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '.version_select at 0x7feb1f68c9d0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:26.219154 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-028a6194-e7e4-4b43-8cb2-cb10fe7f601a tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-107055024" status: 202 len: 0 microversion: 2.1 time: 0.152017 May 28 03:42:26.220377 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 459/886] 10.222.0.41 () {58 vars in 1217 bytes} [Sat May 28 03:42:26 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-107055024 => generated 0 bytes in 154 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:42:26.342907 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4d3ee4b6-f66c-4643-b6fd-3c70417c017b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:26.343665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4d3ee4b6-f66c-4643-b6fd-3c70417c017b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:26.349909 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d3ee4b6-f66c-4643-b6fd-3c70417c017b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:26.350623 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d3ee4b6-f66c-4643-b6fd-3c70417c017b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:26.351388 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d3ee4b6-f66c-4643-b6fd-3c70417c017b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:26.351825 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4d3ee4b6-f66c-4643-b6fd-3c70417c017b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:26.356133 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=90,nova_cell1:INSERT=5,nova_cell1:UPDATE=5 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:26.527919 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4d3ee4b6-f66c-4643-b6fd-3c70417c017b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:26.535966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4d3ee4b6-f66c-4643-b6fd-3c70417c017b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.197702 May 28 03:42:26.545930 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 428/887] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:26 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:26.890400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b503e0c2-663e-4a92-8c8d-1253a2708e6c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:26.891709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b503e0c2-663e-4a92-8c8d-1253a2708e6c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:26.898831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b503e0c2-663e-4a92-8c8d-1253a2708e6c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:26.899276 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b503e0c2-663e-4a92-8c8d-1253a2708e6c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:26.900046 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b503e0c2-663e-4a92-8c8d-1253a2708e6c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:26.900491 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b503e0c2-663e-4a92-8c8d-1253a2708e6c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:27.208799 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b503e0c2-663e-4a92-8c8d-1253a2708e6c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:27.232961 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b503e0c2-663e-4a92-8c8d-1253a2708e6c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:27.234549 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b503e0c2-663e-4a92-8c8d-1253a2708e6c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:27.235779 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b503e0c2-663e-4a92-8c8d-1253a2708e6c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.350137 May 28 03:42:27.236305 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 460/888] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:26 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:27.555044 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-381c871f-37a2-4cd9-90bf-e02528826dfc tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:27.555044 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-381c871f-37a2-4cd9-90bf-e02528826dfc tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:27.565675 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-381c871f-37a2-4cd9-90bf-e02528826dfc tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:27.575396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-381c871f-37a2-4cd9-90bf-e02528826dfc tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:27.584773 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-381c871f-37a2-4cd9-90bf-e02528826dfc tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:27.586709 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-381c871f-37a2-4cd9-90bf-e02528826dfc tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:27.807696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-381c871f-37a2-4cd9-90bf-e02528826dfc tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:27.821229 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-381c871f-37a2-4cd9-90bf-e02528826dfc tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.271939 May 28 03:42:27.821830 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 429/889] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:27 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:28.255646 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e6d070d9-908a-4d85-9095-87cfdc07837b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:28.259052 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e6d070d9-908a-4d85-9095-87cfdc07837b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:28.268354 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e6d070d9-908a-4d85-9095-87cfdc07837b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:28.268812 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e6d070d9-908a-4d85-9095-87cfdc07837b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:28.269619 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e6d070d9-908a-4d85-9095-87cfdc07837b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:28.270206 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e6d070d9-908a-4d85-9095-87cfdc07837b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:28.465061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e6d070d9-908a-4d85-9095-87cfdc07837b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:28.476862 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e6d070d9-908a-4d85-9095-87cfdc07837b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:28.478405 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e6d070d9-908a-4d85-9095-87cfdc07837b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:28.480162 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e6d070d9-908a-4d85-9095-87cfdc07837b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.229068 May 28 03:42:28.493958 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 461/890] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:28 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:28.839668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d81ffbcf-a179-4fce-b19f-a44572a9edd7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:28.840291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d81ffbcf-a179-4fce-b19f-a44572a9edd7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:28.848308 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d81ffbcf-a179-4fce-b19f-a44572a9edd7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:28.848762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d81ffbcf-a179-4fce-b19f-a44572a9edd7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:28.852760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d81ffbcf-a179-4fce-b19f-a44572a9edd7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:28.853772 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d81ffbcf-a179-4fce-b19f-a44572a9edd7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:28.878355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-dd630928-286b-492a-ad52-af851bbf59f9 req-c7554204-4b8a-48da-8a9d-c628cf1707d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b53491f6-a319-4950-ac9e-a9b0bd7a45e2", "name": "network-vif-plugged", "status": "completed", "tag": "ce3985bf-d85c-4d8a-be10-2df6ad569608"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:28.893750 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-dd630928-286b-492a-ad52-af851bbf59f9 req-c7554204-4b8a-48da-8a9d-c628cf1707d8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:28.896283 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-dd630928-286b-492a-ad52-af851bbf59f9 req-c7554204-4b8a-48da-8a9d-c628cf1707d8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:28.930221 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-dd630928-286b-492a-ad52-af851bbf59f9 req-c7554204-4b8a-48da-8a9d-c628cf1707d8 service nova] Creating event network-vif-plugged:ce3985bf-d85c-4d8a-be10-2df6ad569608 for instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 on hv-cinder-84336 May 28 03:42:28.940503 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-dd630928-286b-492a-ad52-af851bbf59f9 req-c7554204-4b8a-48da-8a9d-c628cf1707d8 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065972 May 28 03:42:28.941803 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 462/891] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:42:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:42:29.144141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d81ffbcf-a179-4fce-b19f-a44572a9edd7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:29.175275 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d81ffbcf-a179-4fce-b19f-a44572a9edd7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.330513 May 28 03:42:29.177078 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 430/892] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:28 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:29.505062 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2d529de0-8fbd-4c4d-8ceb-29dbfc45a479 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:29.507499 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2d529de0-8fbd-4c4d-8ceb-29dbfc45a479 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:29.517962 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2d529de0-8fbd-4c4d-8ceb-29dbfc45a479 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:29.518511 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2d529de0-8fbd-4c4d-8ceb-29dbfc45a479 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:29.519431 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2d529de0-8fbd-4c4d-8ceb-29dbfc45a479 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:29.519875 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2d529de0-8fbd-4c4d-8ceb-29dbfc45a479 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:29.688335 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-2d529de0-8fbd-4c4d-8ceb-29dbfc45a479 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:29.698658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-2d529de0-8fbd-4c4d-8ceb-29dbfc45a479 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:29.701091 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-2d529de0-8fbd-4c4d-8ceb-29dbfc45a479 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:29.709258 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2d529de0-8fbd-4c4d-8ceb-29dbfc45a479 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.208510 May 28 03:42:29.710716 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 463/893] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:29 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:30.166929 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3e25d310-ca63-4b22-837b-15f38014ed13 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:30.167526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3e25d310-ca63-4b22-837b-15f38014ed13 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:30.174005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3e25d310-ca63-4b22-837b-15f38014ed13 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:30.174456 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3e25d310-ca63-4b22-837b-15f38014ed13 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:30.175232 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3e25d310-ca63-4b22-837b-15f38014ed13 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:30.175696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3e25d310-ca63-4b22-837b-15f38014ed13 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:30.190710 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-3e25d310-ca63-4b22-837b-15f38014ed13 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] HTTP exception thrown: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3e25d310-ca63-4b22-837b-15f38014ed13 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Request method failure captured: May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: DELETE /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd HTTP/1.1 May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZiiQypFQQ1bdISGQo9C2YguYEEU-tBqQGVthYz_izW7K5gHqUD5gE5Vtsn9MQmMxa3fO2B2uT4EMCIaBoWDOcTCa_6QL577b2i5ZTpp-2FFBn6eD7fm9RJIlmaKR4ZpoEyCHy7RVgamveJZw6aiAUZ4N9PvH67OUDmLYqMGBctoWpU May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:42:30.192355 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: dc6a3335562448d1934fa18e8f51b119 May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:30.193763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '43942790-5855-4ed1-9ce4-8f02a3092edd'} May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:30.194540 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:30.196088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:30.197555 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3e25d310-ca63-4b22-837b-15f38014ed13 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Returning 404 to user: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:30.197555 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3e25d310-ca63-4b22-837b-15f38014ed13 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "DELETE /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 404 len: 111 microversion: 2.1 time: 0.031945 May 28 03:42:30.201127 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-fdd08465-eeeb-4eb5-a6e6-03e9e7cac8ce tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:30.201238 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 431/894] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:42:30 2022] DELETE /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 28 03:42:30.201328 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-fdd08465-eeeb-4eb5-a6e6-03e9e7cac8ce tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:30.203806 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-864285ff-6e56-4d3f-9f29-ba958e5853c8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:30.204830 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-864285ff-6e56-4d3f-9f29-ba958e5853c8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: 43942790-5855-4ed1-9ce4-8f02a3092edd] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:30.206089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fdd08465-eeeb-4eb5-a6e6-03e9e7cac8ce tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:30.206580 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fdd08465-eeeb-4eb5-a6e6-03e9e7cac8ce tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:30.207346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fdd08465-eeeb-4eb5-a6e6-03e9e7cac8ce tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:30.207792 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-fdd08465-eeeb-4eb5-a6e6-03e9e7cac8ce tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:30.212202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-864285ff-6e56-4d3f-9f29-ba958e5853c8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:30.212973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-864285ff-6e56-4d3f-9f29-ba958e5853c8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:30.214130 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-864285ff-6e56-4d3f-9f29-ba958e5853c8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:30.214879 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-864285ff-6e56-4d3f-9f29-ba958e5853c8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:30.235684 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-864285ff-6e56-4d3f-9f29-ba958e5853c8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] HTTP exception thrown: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-864285ff-6e56-4d3f-9f29-ba958e5853c8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Request method failure captured: May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd HTTP/1.1 May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZiiQypFQQ1bdISGQo9C2YguYEEU-tBqQGVthYz_izW7K5gHqUD5gE5Vtsn9MQmMxa3fO2B2uT4EMCIaBoWDOcTCa_6QL577b2i5ZTpp-2FFBn6eD7fm9RJIlmaKR4ZpoEyCHy7RVgamveJZw6aiAUZ4N9PvH67OUDmLYqMGBctoWpU May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:42:30.237267 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: dc6a3335562448d1934fa18e8f51b119 May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:30.240809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '43942790-5855-4ed1-9ce4-8f02a3092edd'} May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:30.242291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. May 28 03:42:30.244222 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:30.245743 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-864285ff-6e56-4d3f-9f29-ba958e5853c8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Returning 404 to user: Instance 43942790-5855-4ed1-9ce4-8f02a3092edd could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:30.245743 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-864285ff-6e56-4d3f-9f29-ba958e5853c8 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd" status: 404 len: 111 microversion: 2.1 time: 0.039719 May 28 03:42:30.245743 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 432/895] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:30 2022] GET /compute/v2.1/servers/43942790-5855-4ed1-9ce4-8f02a3092edd => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:42:30.404910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-fdd08465-eeeb-4eb5-a6e6-03e9e7cac8ce tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:30.412863 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-fdd08465-eeeb-4eb5-a6e6-03e9e7cac8ce tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.223737 May 28 03:42:30.423505 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 464/896] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:30 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:30.733469 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b8de1b2a-d211-4f03-a2c1-1f1086b0285b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:30.734541 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-b8de1b2a-d211-4f03-a2c1-1f1086b0285b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:30.747969 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b8de1b2a-d211-4f03-a2c1-1f1086b0285b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:30.748495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b8de1b2a-d211-4f03-a2c1-1f1086b0285b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:30.749408 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b8de1b2a-d211-4f03-a2c1-1f1086b0285b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:30.749923 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b8de1b2a-d211-4f03-a2c1-1f1086b0285b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:30.971605 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-b8de1b2a-d211-4f03-a2c1-1f1086b0285b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:30.981416 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-b8de1b2a-d211-4f03-a2c1-1f1086b0285b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:30.982912 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-b8de1b2a-d211-4f03-a2c1-1f1086b0285b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:30.984085 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-b8de1b2a-d211-4f03-a2c1-1f1086b0285b tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.256032 May 28 03:42:30.984649 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 433/897] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:30 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:31.431860 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-90ab93a9-e717-4939-a1b8-22a745ba4651 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:31.432552 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-90ab93a9-e717-4939-a1b8-22a745ba4651 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:31.439575 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90ab93a9-e717-4939-a1b8-22a745ba4651 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:31.440363 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90ab93a9-e717-4939-a1b8-22a745ba4651 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:31.441497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90ab93a9-e717-4939-a1b8-22a745ba4651 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:31.442175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-90ab93a9-e717-4939-a1b8-22a745ba4651 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:31.613788 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-90ab93a9-e717-4939-a1b8-22a745ba4651 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:31.623664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-90ab93a9-e717-4939-a1b8-22a745ba4651 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.196269 May 28 03:42:31.624307 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 465/898] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:31 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:32.014983 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e326dfc2-ec7d-4aa1-bae5-731688c12afc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:32.014983 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e326dfc2-ec7d-4aa1-bae5-731688c12afc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:32.023252 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e326dfc2-ec7d-4aa1-bae5-731688c12afc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:32.024024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e326dfc2-ec7d-4aa1-bae5-731688c12afc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:32.025017 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e326dfc2-ec7d-4aa1-bae5-731688c12afc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:32.025692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e326dfc2-ec7d-4aa1-bae5-731688c12afc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:32.187171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e326dfc2-ec7d-4aa1-bae5-731688c12afc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:32.202663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e326dfc2-ec7d-4aa1-bae5-731688c12afc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:32.204868 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e326dfc2-ec7d-4aa1-bae5-731688c12afc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:32.206602 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e326dfc2-ec7d-4aa1-bae5-731688c12afc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.207587 May 28 03:42:32.207677 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 434/899] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:31 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:32.455194 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-83efb333-2fa8-4131-8374-d3bcde96e1d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:32.455763 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-83efb333-2fa8-4131-8374-d3bcde96e1d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:32.464458 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-83efb333-2fa8-4131-8374-d3bcde96e1d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:32.466194 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-83efb333-2fa8-4131-8374-d3bcde96e1d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:32.466194 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-83efb333-2fa8-4131-8374-d3bcde96e1d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:32.466783 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-83efb333-2fa8-4131-8374-d3bcde96e1d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:32.493277 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-83efb333-2fa8-4131-8374-d3bcde96e1d2 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/ips" status: 200 len: 149 microversion: 2.1 time: 0.041776 May 28 03:42:32.500458 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 466/900] 10.222.0.41 () {58 vars in 1247 bytes} [Sat May 28 03:42:32 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/ips => generated 149 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:42:32.642982 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8db47623-95c4-4722-ab6b-d8bb18dde189 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:32.644018 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8db47623-95c4-4722-ab6b-d8bb18dde189 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:32.652317 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8db47623-95c4-4722-ab6b-d8bb18dde189 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:32.653881 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8db47623-95c4-4722-ab6b-d8bb18dde189 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:32.654906 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8db47623-95c4-4722-ab6b-d8bb18dde189 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:32.655561 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8db47623-95c4-4722-ab6b-d8bb18dde189 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:32.672953 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c4afa26b-cd3a-45c2-95a1-0c95bd4e5da3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:32.677756 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c4afa26b-cd3a-45c2-95a1-0c95bd4e5da3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:32.684771 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4afa26b-cd3a-45c2-95a1-0c95bd4e5da3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:32.685719 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4afa26b-cd3a-45c2-95a1-0c95bd4e5da3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:32.687720 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4afa26b-cd3a-45c2-95a1-0c95bd4e5da3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:32.688398 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c4afa26b-cd3a-45c2-95a1-0c95bd4e5da3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:32.718112 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-c4afa26b-cd3a-45c2-95a1-0c95bd4e5da3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] HTTP exception thrown: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c4afa26b-cd3a-45c2-95a1-0c95bd4e5da3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Request method failure captured: May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 HTTP/1.1 May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZiiQypFQQ1bdISGQo9C2YguYEEU-tBqQGVthYz_izW7K5gHqUD5gE5Vtsn9MQmMxa3fO2B2uT4EMCIaBoWDOcTCa_6QL577b2i5ZTpp-2FFBn6eD7fm9RJIlmaKR4ZpoEyCHy7RVgamveJZw6aiAUZ4N9PvH67OUDmLYqMGBctoWpU May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:42:32.719848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: dc6a3335562448d1934fa18e8f51b119 May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:42:32.721605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': 'da053642-eac6-4484-9557-f0b878712cf2'} May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:32.722400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:42:32.723685 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:32.726664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c4afa26b-cd3a-45c2-95a1-0c95bd4e5da3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Returning 404 to user: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:32.727799 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c4afa26b-cd3a-45c2-95a1-0c95bd4e5da3 tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "DELETE /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 404 len: 111 microversion: 2.1 time: 0.059023 May 28 03:42:32.728919 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 467/901] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:42:32 2022] DELETE /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 28 03:42:32.737502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8c353e29-a62f-4ff2-9f38-4bff139f601c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:32.738684 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8c353e29-a62f-4ff2-9f38-4bff139f601c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] [instance: da053642-eac6-4484-9557-f0b878712cf2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:32.747174 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8c353e29-a62f-4ff2-9f38-4bff139f601c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:32.747977 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8c353e29-a62f-4ff2-9f38-4bff139f601c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:32.752522 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8c353e29-a62f-4ff2-9f38-4bff139f601c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:32.752522 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8c353e29-a62f-4ff2-9f38-4bff139f601c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:32.766318 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-8c353e29-a62f-4ff2-9f38-4bff139f601c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] HTTP exception thrown: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8c353e29-a62f-4ff2-9f38-4bff139f601c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Request method failure captured: May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 HTTP/1.1 May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZiiQypFQQ1bdISGQo9C2YguYEEU-tBqQGVthYz_izW7K5gHqUD5gE5Vtsn9MQmMxa3fO2B2uT4EMCIaBoWDOcTCa_6QL577b2i5ZTpp-2FFBn6eD7fm9RJIlmaKR4ZpoEyCHy7RVgamveJZw6aiAUZ4N9PvH67OUDmLYqMGBctoWpU May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:42:32.768213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_70eaf602c8db4ec1bfb26e5eb3b13c82"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 70eaf602c8db4ec1bfb26e5eb3b13c82 May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1322958216 May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: dc6a3335562448d1934fa18e8f51b119 May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeTestJSON-1322958216-project May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:42:32.769562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': 'da053642-eac6-4484-9557-f0b878712cf2'} May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:32.770317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. May 28 03:42:32.771537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:32.774427 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8c353e29-a62f-4ff2-9f38-4bff139f601c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] Returning 404 to user: Instance da053642-eac6-4484-9557-f0b878712cf2 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:32.775883 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8c353e29-a62f-4ff2-9f38-4bff139f601c tempest-AttachVolumeTestJSON-1322958216 tempest-AttachVolumeTestJSON-1322958216-project] 10.222.0.41 "GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2" status: 404 len: 111 microversion: 2.1 time: 0.042608 May 28 03:42:32.777258 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 468/902] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:32 2022] GET /compute/v2.1/servers/da053642-eac6-4484-9557-f0b878712cf2 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:42:32.893981 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8db47623-95c4-4722-ab6b-d8bb18dde189 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:32.907605 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8db47623-95c4-4722-ab6b-d8bb18dde189 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.271313 May 28 03:42:32.919860 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 435/903] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:32 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:33.225274 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-22017d6f-9a1a-4fdb-9768-b22e67fc89ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:33.231871 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-22017d6f-9a1a-4fdb-9768-b22e67fc89ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:33.239115 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-22017d6f-9a1a-4fdb-9768-b22e67fc89ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:33.239731 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-22017d6f-9a1a-4fdb-9768-b22e67fc89ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:33.240745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-22017d6f-9a1a-4fdb-9768-b22e67fc89ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:33.242617 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-22017d6f-9a1a-4fdb-9768-b22e67fc89ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:33.534216 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-22017d6f-9a1a-4fdb-9768-b22e67fc89ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:33.555358 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-22017d6f-9a1a-4fdb-9768-b22e67fc89ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:33.559859 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-22017d6f-9a1a-4fdb-9768-b22e67fc89ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:33.562116 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-22017d6f-9a1a-4fdb-9768-b22e67fc89ca tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.341373 May 28 03:42:33.572869 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 469/904] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:33 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:33.927708 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-efb55672-f55b-442b-8ebf-c9a4105dc47f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:33.928426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-efb55672-f55b-442b-8ebf-c9a4105dc47f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:33.936509 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-efb55672-f55b-442b-8ebf-c9a4105dc47f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:33.936951 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-efb55672-f55b-442b-8ebf-c9a4105dc47f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:33.937765 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-efb55672-f55b-442b-8ebf-c9a4105dc47f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:33.940297 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-efb55672-f55b-442b-8ebf-c9a4105dc47f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:34.175067 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-efb55672-f55b-442b-8ebf-c9a4105dc47f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:34.185296 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-efb55672-f55b-442b-8ebf-c9a4105dc47f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.263669 May 28 03:42:34.187591 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 436/905] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:33 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:34.273791 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d2eae30c-d1a7-4e43-8a04-cb17e256c8cc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7f2c6847-2e31-4038-9bec-f6937eedeec3"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:34.277327 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d2eae30c-d1a7-4e43-8a04-cb17e256c8cc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:34.285634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2eae30c-d1a7-4e43-8a04-cb17e256c8cc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:34.287246 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2eae30c-d1a7-4e43-8a04-cb17e256c8cc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:34.290167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2eae30c-d1a7-4e43-8a04-cb17e256c8cc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:34.290678 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d2eae30c-d1a7-4e43-8a04-cb17e256c8cc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:34.581821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-55dcf1d8-9ccf-4e98-84f1-35dd57edc526 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:34.582518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-55dcf1d8-9ccf-4e98-84f1-35dd57edc526 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:34.589373 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-55dcf1d8-9ccf-4e98-84f1-35dd57edc526 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:34.589891 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-55dcf1d8-9ccf-4e98-84f1-35dd57edc526 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:34.590739 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-55dcf1d8-9ccf-4e98-84f1-35dd57edc526 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:34.591357 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-55dcf1d8-9ccf-4e98-84f1-35dd57edc526 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:34.772175 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-55dcf1d8-9ccf-4e98-84f1-35dd57edc526 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:34.782150 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-55dcf1d8-9ccf-4e98-84f1-35dd57edc526 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:34.785512 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-55dcf1d8-9ccf-4e98-84f1-35dd57edc526 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:34.785512 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-55dcf1d8-9ccf-4e98-84f1-35dd57edc526 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.207400 May 28 03:42:34.785769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 437/906] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:34 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:42:35.204220 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6965829d-a700-4dd9-b51c-a92040e4510a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:35.205551 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6965829d-a700-4dd9-b51c-a92040e4510a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:35.213296 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6965829d-a700-4dd9-b51c-a92040e4510a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:35.213956 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6965829d-a700-4dd9-b51c-a92040e4510a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:35.215320 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6965829d-a700-4dd9-b51c-a92040e4510a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:35.215923 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6965829d-a700-4dd9-b51c-a92040e4510a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:35.461027 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6965829d-a700-4dd9-b51c-a92040e4510a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:35.470533 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6965829d-a700-4dd9-b51c-a92040e4510a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.270087 May 28 03:42:35.471522 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 438/907] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:35 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:42:35.698372 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:INSERT=1,nova_cell1:SELECT=98,nova_cell1:UPDATE=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:35.807353 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-78a8e950-9b06-43ab-afa7-e0b650e01617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:35.808180 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-78a8e950-9b06-43ab-afa7-e0b650e01617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:35.835291 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78a8e950-9b06-43ab-afa7-e0b650e01617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:35.841129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78a8e950-9b06-43ab-afa7-e0b650e01617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:35.845653 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78a8e950-9b06-43ab-afa7-e0b650e01617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:35.846361 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-78a8e950-9b06-43ab-afa7-e0b650e01617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:36.302156 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-78a8e950-9b06-43ab-afa7-e0b650e01617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:36.311911 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-78a8e950-9b06-43ab-afa7-e0b650e01617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:36.313487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-78a8e950-9b06-43ab-afa7-e0b650e01617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:36.314684 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-78a8e950-9b06-43ab-afa7-e0b650e01617 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.514663 May 28 03:42:36.315384 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 439/908] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:35 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:42:36.368283 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d2eae30c-d1a7-4e43-8a04-cb17e256c8cc tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "POST /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.098550 May 28 03:42:36.369395 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 470/909] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:42:34 2022] POST /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/os-volume_attachments => generated 194 bytes in 2101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:42:36.488525 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9eae00f6-bf67-49d1-945e-411e9f4929c2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:36.489262 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9eae00f6-bf67-49d1-945e-411e9f4929c2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:36.496341 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9eae00f6-bf67-49d1-945e-411e9f4929c2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:36.496992 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9eae00f6-bf67-49d1-945e-411e9f4929c2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:36.498046 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9eae00f6-bf67-49d1-945e-411e9f4929c2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:36.499656 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9eae00f6-bf67-49d1-945e-411e9f4929c2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:37.074466 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9eae00f6-bf67-49d1-945e-411e9f4929c2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:37.085769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9eae00f6-bf67-49d1-945e-411e9f4929c2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1267 microversion: 2.1 time: 0.600696 May 28 03:42:37.086964 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 440/910] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:36 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1267 bytes in 603 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:37.334418 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9e0ed997-2a87-4fdd-8a7c-42c55e9f3931 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:37.335160 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9e0ed997-2a87-4fdd-8a7c-42c55e9f3931 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:37.343476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9e0ed997-2a87-4fdd-8a7c-42c55e9f3931 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:37.346132 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9e0ed997-2a87-4fdd-8a7c-42c55e9f3931 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:37.346132 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9e0ed997-2a87-4fdd-8a7c-42c55e9f3931 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:37.346132 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9e0ed997-2a87-4fdd-8a7c-42c55e9f3931 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:37.649535 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9e0ed997-2a87-4fdd-8a7c-42c55e9f3931 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:37.661572 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9e0ed997-2a87-4fdd-8a7c-42c55e9f3931 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:37.663545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9e0ed997-2a87-4fdd-8a7c-42c55e9f3931 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:37.664777 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9e0ed997-2a87-4fdd-8a7c-42c55e9f3931 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.335226 May 28 03:42:37.665487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 471/911] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:37 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:38.105134 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-98fff961-048f-47d1-aaab-63a411eca0d8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:38.108024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-98fff961-048f-47d1-aaab-63a411eca0d8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:38.117793 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-98fff961-048f-47d1-aaab-63a411eca0d8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:38.118317 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-98fff961-048f-47d1-aaab-63a411eca0d8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:38.119432 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-98fff961-048f-47d1-aaab-63a411eca0d8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:38.120243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-98fff961-048f-47d1-aaab-63a411eca0d8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:38.319911 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-98fff961-048f-47d1-aaab-63a411eca0d8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:38.329665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-98fff961-048f-47d1-aaab-63a411eca0d8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1453 microversion: 2.1 time: 0.230199 May 28 03:42:38.330233 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 441/912] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:38 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1453 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:38.355937 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-06ddd5c6-ff14-43f1-8079-8c701b25760b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:38.356562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-06ddd5c6-ff14-43f1-8079-8c701b25760b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:38.367582 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06ddd5c6-ff14-43f1-8079-8c701b25760b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:38.367939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06ddd5c6-ff14-43f1-8079-8c701b25760b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:38.370910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06ddd5c6-ff14-43f1-8079-8c701b25760b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:38.371442 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-06ddd5c6-ff14-43f1-8079-8c701b25760b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:38.614323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-06ddd5c6-ff14-43f1-8079-8c701b25760b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:38.627541 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-06ddd5c6-ff14-43f1-8079-8c701b25760b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1453 microversion: 2.1 time: 0.272751 May 28 03:42:38.627541 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 472/913] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:38 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1453 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:38.687343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cbbe9ae4-3f5f-47fd-9d2a-5fb7bc2febdc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:38.688224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-cbbe9ae4-3f5f-47fd-9d2a-5fb7bc2febdc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:38.696821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cbbe9ae4-3f5f-47fd-9d2a-5fb7bc2febdc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:38.697524 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cbbe9ae4-3f5f-47fd-9d2a-5fb7bc2febdc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:38.698530 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cbbe9ae4-3f5f-47fd-9d2a-5fb7bc2febdc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:38.699165 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cbbe9ae4-3f5f-47fd-9d2a-5fb7bc2febdc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:38.928342 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-cbbe9ae4-3f5f-47fd-9d2a-5fb7bc2febdc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:38.937875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-cbbe9ae4-3f5f-47fd-9d2a-5fb7bc2febdc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:38.939645 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-cbbe9ae4-3f5f-47fd-9d2a-5fb7bc2febdc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:38.949552 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-cbbe9ae4-3f5f-47fd-9d2a-5fb7bc2febdc tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.259137 May 28 03:42:38.949552 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 442/914] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:38 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:39.963287 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4dcdad85-be21-427a-a691-7674c118305c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:39.964757 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4dcdad85-be21-427a-a691-7674c118305c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:39.973413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4dcdad85-be21-427a-a691-7674c118305c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:39.973413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4dcdad85-be21-427a-a691-7674c118305c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:39.974293 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4dcdad85-be21-427a-a691-7674c118305c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:39.974877 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4dcdad85-be21-427a-a691-7674c118305c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:40.310412 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4dcdad85-be21-427a-a691-7674c118305c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:40.319414 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4dcdad85-be21-427a-a691-7674c118305c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:40.322303 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4dcdad85-be21-427a-a691-7674c118305c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:40.323344 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4dcdad85-be21-427a-a691-7674c118305c tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.366897 May 28 03:42:40.324374 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 473/915] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:39 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:41.029594 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-34660db2-1f80-472f-b86a-47c030ca96d1 req-a1f239c4-4195-4d38-a9cf-abdf96934bbf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6be2aeaa-8d90-4e51-87d2-d02a8566ac1a", "name": "network-vif-plugged", "status": "completed", "tag": "73f87b73-9d7c-43a7-9bd2-dc7fb5450112"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:41.049673 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-34660db2-1f80-472f-b86a-47c030ca96d1 req-a1f239c4-4195-4d38-a9cf-abdf96934bbf service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:41.050090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-34660db2-1f80-472f-b86a-47c030ca96d1 req-a1f239c4-4195-4d38-a9cf-abdf96934bbf service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:41.073175 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-34660db2-1f80-472f-b86a-47c030ca96d1 req-a1f239c4-4195-4d38-a9cf-abdf96934bbf service nova] Creating event network-vif-plugged:73f87b73-9d7c-43a7-9bd2-dc7fb5450112 for instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a on hv-cinder-84336 May 28 03:42:41.080881 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-34660db2-1f80-472f-b86a-47c030ca96d1 req-a1f239c4-4195-4d38-a9cf-abdf96934bbf service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058391 May 28 03:42:41.082817 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 443/916] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:42:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:42:41.342410 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ae1c1d96-2742-4874-8763-0c9b3533f3ea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:41.343832 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ae1c1d96-2742-4874-8763-0c9b3533f3ea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:41.351176 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ae1c1d96-2742-4874-8763-0c9b3533f3ea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:41.351632 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ae1c1d96-2742-4874-8763-0c9b3533f3ea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:41.352418 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ae1c1d96-2742-4874-8763-0c9b3533f3ea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:41.353053 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ae1c1d96-2742-4874-8763-0c9b3533f3ea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:41.560297 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ae1c1d96-2742-4874-8763-0c9b3533f3ea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:41.568975 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ae1c1d96-2742-4874-8763-0c9b3533f3ea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:41.571061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ae1c1d96-2742-4874-8763-0c9b3533f3ea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:41.572599 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ae1c1d96-2742-4874-8763-0c9b3533f3ea tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.235249 May 28 03:42:41.589626 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 474/917] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:41 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:41.797760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-ad2a82d2-69ca-43a3-a3b7-84a2edb67b9c req-e54021ed-6be4-498b-aab6-8fbe651a6ae6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b53491f6-a319-4950-ac9e-a9b0bd7a45e2", "tag": "ce3985bf-d85c-4d8a-be10-2df6ad569608"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:41.811002 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-ad2a82d2-69ca-43a3-a3b7-84a2edb67b9c req-e54021ed-6be4-498b-aab6-8fbe651a6ae6 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:41.811817 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-ad2a82d2-69ca-43a3-a3b7-84a2edb67b9c req-e54021ed-6be4-498b-aab6-8fbe651a6ae6 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:41.831517 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-ad2a82d2-69ca-43a3-a3b7-84a2edb67b9c req-e54021ed-6be4-498b-aab6-8fbe651a6ae6 service nova] Creating event network-changed:ce3985bf-d85c-4d8a-be10-2df6ad569608 for instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 on hv-cinder-84336 May 28 03:42:41.862076 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-ad2a82d2-69ca-43a3-a3b7-84a2edb67b9c req-e54021ed-6be4-498b-aab6-8fbe651a6ae6 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069825 May 28 03:42:41.863914 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 444/918] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:42:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:42:42.591779 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e2b312ac-dae4-41b7-af85-6986073b58c4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:42.593250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e2b312ac-dae4-41b7-af85-6986073b58c4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:42.600461 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2b312ac-dae4-41b7-af85-6986073b58c4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:42.600899 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2b312ac-dae4-41b7-af85-6986073b58c4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:42.601887 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2b312ac-dae4-41b7-af85-6986073b58c4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:42.602567 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e2b312ac-dae4-41b7-af85-6986073b58c4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:42.780811 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e2b312ac-dae4-41b7-af85-6986073b58c4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:42.797646 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e2b312ac-dae4-41b7-af85-6986073b58c4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:42.800648 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-e2b312ac-dae4-41b7-af85-6986073b58c4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:42.802519 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e2b312ac-dae4-41b7-af85-6986073b58c4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.214714 May 28 03:42:42.821055 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 475/919] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:42 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:43.823303 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3221d17d-234f-430b-92e2-69e3cba36706 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:43.824203 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3221d17d-234f-430b-92e2-69e3cba36706 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:43.831440 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3221d17d-234f-430b-92e2-69e3cba36706 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:43.831890 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3221d17d-234f-430b-92e2-69e3cba36706 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:43.832665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3221d17d-234f-430b-92e2-69e3cba36706 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:43.833164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3221d17d-234f-430b-92e2-69e3cba36706 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:44.001139 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3221d17d-234f-430b-92e2-69e3cba36706 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:44.009965 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3221d17d-234f-430b-92e2-69e3cba36706 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:44.015542 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3221d17d-234f-430b-92e2-69e3cba36706 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:44.015542 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3221d17d-234f-430b-92e2-69e3cba36706 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.194724 May 28 03:42:44.015542 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 445/920] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:43 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:45.030985 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-da003381-fd62-4fb7-818c-8012fe776e14 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:45.031661 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-da003381-fd62-4fb7-818c-8012fe776e14 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:45.040947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-da003381-fd62-4fb7-818c-8012fe776e14 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:45.041922 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-da003381-fd62-4fb7-818c-8012fe776e14 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:45.042958 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-da003381-fd62-4fb7-818c-8012fe776e14 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:45.043936 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-da003381-fd62-4fb7-818c-8012fe776e14 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:45.461277 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-521073c8-6f30-4f5c-9818-24fddae298ac tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:45.462546 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-521073c8-6f30-4f5c-9818-24fddae298ac tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:45.480664 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-521073c8-6f30-4f5c-9818-24fddae298ac tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:45.480664 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-521073c8-6f30-4f5c-9818-24fddae298ac tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:45.480664 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-521073c8-6f30-4f5c-9818-24fddae298ac tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:45.488182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-521073c8-6f30-4f5c-9818-24fddae298ac tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:45.590982 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-da003381-fd62-4fb7-818c-8012fe776e14 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:45.601427 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-da003381-fd62-4fb7-818c-8012fe776e14 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:45.603656 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-da003381-fd62-4fb7-818c-8012fe776e14 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:45.606004 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-da003381-fd62-4fb7-818c-8012fe776e14 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1731 microversion: 2.20 time: 0.579499 May 28 03:42:45.612584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 476/921] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:45 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1731 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:45.612952 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-521073c8-6f30-4f5c-9818-24fddae298ac tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lazy-loading 'services' on Instance uuid f65cf50d-baf3-4a38-9dec-14ff91d5f6ac {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:42:45.846845 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:45.968813 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-521073c8-6f30-4f5c-9818-24fddae298ac tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/os-volume_attachments/7f2c6847-2e31-4038-9bec-f6937eedeec3" status: 202 len: 0 microversion: 2.1 time: 0.517278 May 28 03:42:45.969656 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 446/922] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:42:45 2022] DELETE /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/os-volume_attachments/7f2c6847-2e31-4038-9bec-f6937eedeec3 => generated 0 bytes in 520 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 28 03:42:45.978500 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2413f48d-f802-4ac5-941a-a6a5f0d66c0c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:45.979506 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2413f48d-f802-4ac5-941a-a6a5f0d66c0c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:45.988706 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2413f48d-f802-4ac5-941a-a6a5f0d66c0c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:45.989227 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2413f48d-f802-4ac5-941a-a6a5f0d66c0c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:45.990011 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2413f48d-f802-4ac5-941a-a6a5f0d66c0c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:45.990439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2413f48d-f802-4ac5-941a-a6a5f0d66c0c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:46.047311 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2413f48d-f802-4ac5-941a-a6a5f0d66c0c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073187 May 28 03:42:46.052935 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 477/923] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:42:45 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/os-volume_attachments => generated 197 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:42:46.632026 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7b624ab0-1b2c-48e5-8ff2-27aabc810b0e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:46.633705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7b624ab0-1b2c-48e5-8ff2-27aabc810b0e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:46.653736 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7b624ab0-1b2c-48e5-8ff2-27aabc810b0e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:46.655586 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7b624ab0-1b2c-48e5-8ff2-27aabc810b0e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:46.655586 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7b624ab0-1b2c-48e5-8ff2-27aabc810b0e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:46.655586 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7b624ab0-1b2c-48e5-8ff2-27aabc810b0e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:46.841421 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7b624ab0-1b2c-48e5-8ff2-27aabc810b0e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:46.855706 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7b624ab0-1b2c-48e5-8ff2-27aabc810b0e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:46.857982 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7b624ab0-1b2c-48e5-8ff2-27aabc810b0e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:46.860262 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7b624ab0-1b2c-48e5-8ff2-27aabc810b0e tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1718 microversion: 2.20 time: 0.238810 May 28 03:42:46.861650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 447/924] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:42:46 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1718 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:42:47.061891 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a2583509-1d67-4d03-be0b-68f7ed3ef148 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:47.063507 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a2583509-1d67-4d03-be0b-68f7ed3ef148 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:47.078714 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2583509-1d67-4d03-be0b-68f7ed3ef148 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:47.078714 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2583509-1d67-4d03-be0b-68f7ed3ef148 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:47.078714 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2583509-1d67-4d03-be0b-68f7ed3ef148 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:47.078714 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a2583509-1d67-4d03-be0b-68f7ed3ef148 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:47.145583 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a2583509-1d67-4d03-be0b-68f7ed3ef148 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.088446 May 28 03:42:47.146159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 478/925] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:42:47 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac/os-volume_attachments => generated 25 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 28 03:42:47.231849 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e550860b-28dd-40ad-bda8-ac2d342c9445 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:47.232641 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e550860b-28dd-40ad-bda8-ac2d342c9445 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:47.240504 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e550860b-28dd-40ad-bda8-ac2d342c9445 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:47.245056 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e550860b-28dd-40ad-bda8-ac2d342c9445 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:47.245056 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e550860b-28dd-40ad-bda8-ac2d342c9445 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:47.245056 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e550860b-28dd-40ad-bda8-ac2d342c9445 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:47.272023 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e550860b-28dd-40ad-bda8-ac2d342c9445 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:42:47.562650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e550860b-28dd-40ad-bda8-ac2d342c9445 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 204 len: 0 microversion: 2.1 time: 0.334838 May 28 03:42:47.563446 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 448/926] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:42:47 2022] DELETE /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 0 bytes in 337 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:42:47.574267 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ab72ccff-da04-4895-ade0-3dab11500ab5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:47.579194 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ab72ccff-da04-4895-ade0-3dab11500ab5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:47.582630 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ab72ccff-da04-4895-ade0-3dab11500ab5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:47.583082 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ab72ccff-da04-4895-ade0-3dab11500ab5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:47.583908 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ab72ccff-da04-4895-ade0-3dab11500ab5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:47.584352 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ab72ccff-da04-4895-ade0-3dab11500ab5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:47.864621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ab72ccff-da04-4895-ade0-3dab11500ab5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:47.872668 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ab72ccff-da04-4895-ade0-3dab11500ab5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1681 microversion: 2.1 time: 0.303008 May 28 03:42:47.873325 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 479/927] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:47 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1681 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:48.890397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-122fe1fd-1553-4aa1-afff-1a8d31ae6f7a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:48.891090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-122fe1fd-1553-4aa1-afff-1a8d31ae6f7a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:48.897952 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-122fe1fd-1553-4aa1-afff-1a8d31ae6f7a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:48.898393 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-122fe1fd-1553-4aa1-afff-1a8d31ae6f7a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:48.899604 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-122fe1fd-1553-4aa1-afff-1a8d31ae6f7a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:48.899707 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-122fe1fd-1553-4aa1-afff-1a8d31ae6f7a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:49.042737 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-122fe1fd-1553-4aa1-afff-1a8d31ae6f7a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:49.050427 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-122fe1fd-1553-4aa1-afff-1a8d31ae6f7a tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1681 microversion: 2.1 time: 0.164320 May 28 03:42:49.050996 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 449/928] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:48 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1681 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:50.068035 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-722c76a3-4e27-4a79-89eb-d4e432d56644 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:50.069888 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-722c76a3-4e27-4a79-89eb-d4e432d56644 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:50.073774 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=76,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:50.077262 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-722c76a3-4e27-4a79-89eb-d4e432d56644 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:50.077754 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-722c76a3-4e27-4a79-89eb-d4e432d56644 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:50.078572 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-722c76a3-4e27-4a79-89eb-d4e432d56644 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:50.079006 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-722c76a3-4e27-4a79-89eb-d4e432d56644 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:50.270371 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-722c76a3-4e27-4a79-89eb-d4e432d56644 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:50.284863 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-722c76a3-4e27-4a79-89eb-d4e432d56644 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1681 microversion: 2.1 time: 0.219767 May 28 03:42:50.285407 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 480/929] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:50 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1681 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:51.302975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-fdfc2573-61c0-4b8a-9c25-dd2322ca81f4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:51.304660 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-fdfc2573-61c0-4b8a-9c25-dd2322ca81f4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:51.313096 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fdfc2573-61c0-4b8a-9c25-dd2322ca81f4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:51.315026 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fdfc2573-61c0-4b8a-9c25-dd2322ca81f4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:51.316108 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fdfc2573-61c0-4b8a-9c25-dd2322ca81f4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:51.316763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fdfc2573-61c0-4b8a-9c25-dd2322ca81f4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:51.682011 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-fdfc2573-61c0-4b8a-9c25-dd2322ca81f4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:51.691676 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-fdfc2573-61c0-4b8a-9c25-dd2322ca81f4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1681 microversion: 2.1 time: 0.393064 May 28 03:42:51.705401 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 450/930] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:51 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1681 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:52.239112 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-a074cee4-1048-437b-a83b-f3662653309c req-ae3c04bd-655a-4fde-b819-20d4210d40f3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f65cf50d-baf3-4a38-9dec-14ff91d5f6ac", "name": "network-vif-deleted", "tag": "b60c0498-d47b-4439-b615-1ff9bb9a032a"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:42:52.257125 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-a074cee4-1048-437b-a83b-f3662653309c req-ae3c04bd-655a-4fde-b819-20d4210d40f3 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:52.257571 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-a074cee4-1048-437b-a83b-f3662653309c req-ae3c04bd-655a-4fde-b819-20d4210d40f3 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:52.283853 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-a074cee4-1048-437b-a83b-f3662653309c req-ae3c04bd-655a-4fde-b819-20d4210d40f3 service nova] Creating event network-vif-deleted:b60c0498-d47b-4439-b615-1ff9bb9a032a for instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac on hv-cinder-84336 May 28 03:42:52.291207 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-a074cee4-1048-437b-a83b-f3662653309c req-ae3c04bd-655a-4fde-b819-20d4210d40f3 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055384 May 28 03:42:52.292929 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 481/931] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:42:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:42:52.713561 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ce04baa7-283c-4fe9-968a-8294645b6795 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:52.715105 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ce04baa7-283c-4fe9-968a-8294645b6795 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:52.718014 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=63,nova_api:UPDATE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:52.730323 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ce04baa7-283c-4fe9-968a-8294645b6795 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:52.730792 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ce04baa7-283c-4fe9-968a-8294645b6795 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:52.731584 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ce04baa7-283c-4fe9-968a-8294645b6795 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:52.732020 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ce04baa7-283c-4fe9-968a-8294645b6795 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:52.787227 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ce04baa7-283c-4fe9-968a-8294645b6795 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:52.787710 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-ce04baa7-283c-4fe9-968a-8294645b6795 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:52.788368 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-ce04baa7-283c-4fe9-968a-8294645b6795 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lazy-loading 'fault' on Instance uuid f65cf50d-baf3-4a38-9dec-14ff91d5f6ac {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:42:52.850969 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ce04baa7-283c-4fe9-968a-8294645b6795 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9d420e4b554e4deab21b89094e0547ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68e1a9e256f64e46919b1e362b4f68d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:42:52.859946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ce04baa7-283c-4fe9-968a-8294645b6795 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 200 len: 1332 microversion: 2.1 time: 0.151322 May 28 03:42:52.863432 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 451/932] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:52 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 1332 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:42:53.879736 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e78134c8-4346-4146-a473-572e3bfc7823 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:53.882311 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-e78134c8-4346-4146-a473-572e3bfc7823 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:53.895902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e78134c8-4346-4146-a473-572e3bfc7823 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:53.898349 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e78134c8-4346-4146-a473-572e3bfc7823 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:53.901280 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e78134c8-4346-4146-a473-572e3bfc7823 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:53.937310 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-e78134c8-4346-4146-a473-572e3bfc7823 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.029s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:53.946702 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-e78134c8-4346-4146-a473-572e3bfc7823 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e78134c8-4346-4146-a473-572e3bfc7823 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac HTTP/1.1 May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:42:53.948547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:53.950526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': 'f65cf50d-baf3-4a38-9dec-14ff91d5f6ac'} May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:53.954947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:53.956679 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:53.958159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-e78134c8-4346-4146-a473-572e3bfc7823 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:53.958159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-e78134c8-4346-4146-a473-572e3bfc7823 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 404 len: 111 microversion: 2.1 time: 0.075465 May 28 03:42:53.958159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 482/933] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:53 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 111 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:42:53.959253 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e3efaf01-172e-4239-a858-d6c2eb0cec8e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '.version_select at 0x7feb1fe4b040>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:54.059163 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e3efaf01-172e-4239-a858-d6c2eb0cec8e tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1392860992" status: 202 len: 0 microversion: 2.1 time: 0.103739 May 28 03:42:54.059850 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 452/934] 10.222.0.41 () {58 vars in 1220 bytes} [Sat May 28 03:42:53 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1392860992 => generated 0 bytes in 105 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:42:56.470242 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3c97fbc6-4885-4d4d-8b7b-a267d091e25f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:56.470888 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3c97fbc6-4885-4d4d-8b7b-a267d091e25f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:56.478169 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c97fbc6-4885-4d4d-8b7b-a267d091e25f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:56.478883 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c97fbc6-4885-4d4d-8b7b-a267d091e25f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:56.479923 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c97fbc6-4885-4d4d-8b7b-a267d091e25f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:56.480579 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3c97fbc6-4885-4d4d-8b7b-a267d091e25f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:56.499456 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-3c97fbc6-4885-4d4d-8b7b-a267d091e25f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3c97fbc6-4885-4d4d-8b7b-a267d091e25f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac HTTP/1.1 May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:42:56.501382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:56.502690 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': 'f65cf50d-baf3-4a38-9dec-14ff91d5f6ac'} May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:56.503439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:56.505040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:56.507187 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3c97fbc6-4885-4d4d-8b7b-a267d091e25f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:56.508328 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3c97fbc6-4885-4d4d-8b7b-a267d091e25f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 404 len: 111 microversion: 2.1 time: 0.042522 May 28 03:42:56.512243 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 483/935] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:42:56 2022] DELETE /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:42:56.519330 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b83320aa-0efb-4123-86f1-09192cd986d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:56.519919 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-b83320aa-0efb-4123-86f1-09192cd986d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: f65cf50d-baf3-4a38-9dec-14ff91d5f6ac] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:56.526333 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b83320aa-0efb-4123-86f1-09192cd986d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:56.536184 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b83320aa-0efb-4123-86f1-09192cd986d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:56.536184 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b83320aa-0efb-4123-86f1-09192cd986d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:56.536184 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b83320aa-0efb-4123-86f1-09192cd986d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:56.545713 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-b83320aa-0efb-4123-86f1-09192cd986d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b83320aa-0efb-4123-86f1-09192cd986d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac HTTP/1.1 May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:42:56.547386 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:56.548946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': 'f65cf50d-baf3-4a38-9dec-14ff91d5f6ac'} May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:56.549926 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. May 28 03:42:56.551663 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:56.554444 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b83320aa-0efb-4123-86f1-09192cd986d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance f65cf50d-baf3-4a38-9dec-14ff91d5f6ac could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:56.554444 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-b83320aa-0efb-4123-86f1-09192cd986d5 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac" status: 404 len: 111 microversion: 2.1 time: 0.035183 May 28 03:42:56.554444 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 453/936] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:56 2022] GET /compute/v2.1/servers/f65cf50d-baf3-4a38-9dec-14ff91d5f6ac => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:42:57.929079 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-cdea6b62-f78c-4745-b60d-54e699b12d6c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:57.930165 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-cdea6b62-f78c-4745-b60d-54e699b12d6c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:57.937014 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cdea6b62-f78c-4745-b60d-54e699b12d6c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:57.937722 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cdea6b62-f78c-4745-b60d-54e699b12d6c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:57.938563 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cdea6b62-f78c-4745-b60d-54e699b12d6c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:57.939168 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cdea6b62-f78c-4745-b60d-54e699b12d6c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:57.955585 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-cdea6b62-f78c-4745-b60d-54e699b12d6c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-cdea6b62-f78c-4745-b60d-54e699b12d6c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 HTTP/1.1 May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:42:57.957645 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:42:57.958981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': 'c2d3a613-c1f1-49d5-a707-784bdc430953'} May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:57.960446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:42:57.961800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:57.964752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-cdea6b62-f78c-4745-b60d-54e699b12d6c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:57.965651 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-cdea6b62-f78c-4745-b60d-54e699b12d6c tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 404 len: 111 microversion: 2.1 time: 0.040594 May 28 03:42:57.966240 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 484/937] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:42:57 2022] DELETE /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:42:57.974299 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-86eff350-0204-491b-99e8-c076c50be4c4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:57.975223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-86eff350-0204-491b-99e8-c076c50be4c4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: c2d3a613-c1f1-49d5-a707-784bdc430953] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:57.982239 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-86eff350-0204-491b-99e8-c076c50be4c4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:57.982692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-86eff350-0204-491b-99e8-c076c50be4c4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:57.983479 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-86eff350-0204-491b-99e8-c076c50be4c4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:57.983929 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-86eff350-0204-491b-99e8-c076c50be4c4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:57.998739 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-86eff350-0204-491b-99e8-c076c50be4c4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-86eff350-0204-491b-99e8-c076c50be4c4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 HTTP/1.1 May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:42:58.000461 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:42:58.001954 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': 'c2d3a613-c1f1-49d5-a707-784bdc430953'} May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:58.002705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. May 28 03:42:58.004084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:58.005816 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-86eff350-0204-491b-99e8-c076c50be4c4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance c2d3a613-c1f1-49d5-a707-784bdc430953 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:58.005816 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-86eff350-0204-491b-99e8-c076c50be4c4 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953" status: 404 len: 111 microversion: 2.1 time: 0.031484 May 28 03:42:58.011406 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0c9a92ae-2049-4227-b8e1-0e25138b5571 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:58.016502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0c9a92ae-2049-4227-b8e1-0e25138b5571 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:58.016638 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 454/938] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:57 2022] GET /compute/v2.1/servers/c2d3a613-c1f1-49d5-a707-784bdc430953 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:42:58.018653 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0c9a92ae-2049-4227-b8e1-0e25138b5571 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:58.019296 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0c9a92ae-2049-4227-b8e1-0e25138b5571 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:58.020268 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0c9a92ae-2049-4227-b8e1-0e25138b5571 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:58.020909 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0c9a92ae-2049-4227-b8e1-0e25138b5571 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:58.036957 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-0c9a92ae-2049-4227-b8e1-0e25138b5571 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0c9a92ae-2049-4227-b8e1-0e25138b5571 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 HTTP/1.1 May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:42:58.038745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:42:58.046323 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': '8714c46b-f821-4c0a-b55d-1083da5eb5b8'} May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:58.047149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:42:58.048601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:58.050965 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0c9a92ae-2049-4227-b8e1-0e25138b5571 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:58.050965 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0c9a92ae-2049-4227-b8e1-0e25138b5571 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 404 len: 111 microversion: 2.1 time: 0.033469 May 28 03:42:58.050965 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 485/939] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:42:58 2022] DELETE /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:42:58.052607 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b1bee612-4714-47d4-8925-7e633d89ab1f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:58.055274 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-b1bee612-4714-47d4-8925-7e633d89ab1f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 8714c46b-f821-4c0a-b55d-1083da5eb5b8] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:58.062043 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b1bee612-4714-47d4-8925-7e633d89ab1f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:58.062498 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b1bee612-4714-47d4-8925-7e633d89ab1f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:58.063477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b1bee612-4714-47d4-8925-7e633d89ab1f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:58.063927 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b1bee612-4714-47d4-8925-7e633d89ab1f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:58.088128 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-b1bee612-4714-47d4-8925-7e633d89ab1f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b1bee612-4714-47d4-8925-7e633d89ab1f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 HTTP/1.1 May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:42:58.089967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:42:58.094183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '8714c46b-f821-4c0a-b55d-1083da5eb5b8'} May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:58.095223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. May 28 03:42:58.096782 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:58.098716 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b1bee612-4714-47d4-8925-7e633d89ab1f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance 8714c46b-f821-4c0a-b55d-1083da5eb5b8 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:58.098716 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-b1bee612-4714-47d4-8925-7e633d89ab1f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8" status: 404 len: 111 microversion: 2.1 time: 0.046585 May 28 03:42:58.098716 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 455/940] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:58 2022] GET /compute/v2.1/servers/8714c46b-f821-4c0a-b55d-1083da5eb5b8 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:42:59.362507 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-54c334a2-4347-4589-a05a-dc07b79ed865 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:59.363662 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-54c334a2-4347-4589-a05a-dc07b79ed865 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:59.371433 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-54c334a2-4347-4589-a05a-dc07b79ed865 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:59.371907 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-54c334a2-4347-4589-a05a-dc07b79ed865 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:59.372703 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-54c334a2-4347-4589-a05a-dc07b79ed865 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:59.373160 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-54c334a2-4347-4589-a05a-dc07b79ed865 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:59.390448 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-54c334a2-4347-4589-a05a-dc07b79ed865 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-54c334a2-4347-4589-a05a-dc07b79ed865 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f HTTP/1.1 May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:42:59.392688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:42:59.394312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': '161bde4a-7b5a-4c34-9cad-add4855d9d1f'} May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:59.395136 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:42:59.396389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:42:59.400125 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-54c334a2-4347-4589-a05a-dc07b79ed865 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:59.400125 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-54c334a2-4347-4589-a05a-dc07b79ed865 tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "DELETE /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 404 len: 111 microversion: 2.1 time: 0.037761 May 28 03:42:59.400603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 486/941] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:42:59 2022] DELETE /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:42:59.410382 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5d7a581e-cf70-495a-a309-cc60f2adbb8f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:42:59.412221 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5d7a581e-cf70-495a-a309-cc60f2adbb8f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] [instance: 161bde4a-7b5a-4c34-9cad-add4855d9d1f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:42:59.422297 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5d7a581e-cf70-495a-a309-cc60f2adbb8f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:59.423490 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5d7a581e-cf70-495a-a309-cc60f2adbb8f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:59.427411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5d7a581e-cf70-495a-a309-cc60f2adbb8f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:42:59.427411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5d7a581e-cf70-495a-a309-cc60f2adbb8f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:42:59.442599 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-5d7a581e-cf70-495a-a309-cc60f2adbb8f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] HTTP exception thrown: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5d7a581e-cf70-495a-a309-cc60f2adbb8f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Request method failure captured: May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f HTTP/1.1 May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZihESJzGqGn-lCWpjyhYB4LKPB5GVa9BHiB7rOZ8j58RShn5z4F3eyR8EgMw0w7NQBwh-QXRX4D_gushtiU9pqIrjLKpwPxkstqeHrHa-Sn1NtZPvhz4PTZqRE4p5Jg2JvSbeiotGgorvHp3vcxBO2WcHqRx4jgRwxf7p3_7A6VJMs May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:42:59.445089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_68e1a9e256f64e46919b1e362b4f68d8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 68e1a9e256f64e46919b1e362b4f68d8 May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-969601466 May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 9d420e4b554e4deab21b89094e0547ca May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeNegativeTest-969601466-project May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:42:59.447493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '161bde4a-7b5a-4c34-9cad-add4855d9d1f'} May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:42:59.448757 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. May 28 03:42:59.450724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:42:59.453411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5d7a581e-cf70-495a-a309-cc60f2adbb8f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] Returning 404 to user: Instance 161bde4a-7b5a-4c34-9cad-add4855d9d1f could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:42:59.453411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5d7a581e-cf70-495a-a309-cc60f2adbb8f tempest-AttachVolumeNegativeTest-969601466 tempest-AttachVolumeNegativeTest-969601466-project] 10.222.0.41 "GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f" status: 404 len: 111 microversion: 2.1 time: 0.042836 May 28 03:42:59.453411 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 456/942] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:42:59 2022] GET /compute/v2.1/servers/161bde4a-7b5a-4c34-9cad-add4855d9d1f => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:43:02.009060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b17acfc1-9887-4815-af0d-cbf323951b9a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Action: 'create', calling method: .version_select at 0x7feb1f6f9670>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1616322512", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK4MeDplvE/QnNHutocXbERtrbKS6h+6HnqKSBi7Jph8fPCD6o8/1V+PjWLvcXvH7xlNKFmLCJBUxHDuNG1LHpq2UC7P7eiY0ON78hJ9RsKXTdPiyAOHFKr8V5y1uIgcaw=="}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:02.024205 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-b17acfc1-9887-4815-af0d-cbf323951b9a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Getting quotas for project 23f8194639b14942b36011698807b031. Resources: {'key_pairs'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:43:02.028295 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-b17acfc1-9887-4815-af0d-cbf323951b9a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Getting quotas for user c1a9600bad844f60937aa35126cd1be4 and project 23f8194639b14942b36011698807b031. Resources: {'key_pairs'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:43:02.140180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b17acfc1-9887-4815-af0d-cbf323951b9a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.135304 May 28 03:43:02.141930 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 487/943] 10.222.0.41 () {60 vars in 1158 bytes} [Sat May 28 03:43:02 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:02.199742 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-914151677", "imageRef": "b76127bb-819b-4dda-9e98-e169f8498762", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1616322512", "networks": [{"uuid": "184aa47e-df50-4987-8d3f-0e187a7b314b"}]}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:02.250486 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.network.neutron [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] validate_networks() for [('184aa47e-df50-4987-8d3f-0e187a7b314b', None, None, None, None, None)] {{(pid=124605) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:43:02.634095 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Flavor limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:43:02.634633 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Image limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:43:02.635088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Flavor pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:43:02.635543 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Image pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:43:02.635944 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:43:02.644666 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:02.645122 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:02.645973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:02.656908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:02.663680 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Getting quotas for project 23f8194639b14942b36011698807b031. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:43:02.667526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Getting quotas for user c1a9600bad844f60937aa35126cd1be4 and project 23f8194639b14942b36011698807b031. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:43:02.678722 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Going to run 1 instances... {{(pid=124605) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:43:02.680984 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='b76127bb-819b-4dda-9e98-e169f8498762',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:43:02.822141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-610e06dd-aea7-4088-be99-252aa811119a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.626221 May 28 03:43:02.823024 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 457/944] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:43:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 628 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:43:02.832671 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-457af15e-cbcb-40da-a870-ee747a1dd065 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:02.833330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-457af15e-cbcb-40da-a870-ee747a1dd065 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:02.920188 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-457af15e-cbcb-40da-a870-ee747a1dd065 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:02.933205 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-457af15e-cbcb-40da-a870-ee747a1dd065 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1258 microversion: 2.1 time: 0.102896 May 28 03:43:02.943186 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 488/945] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:02 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1258 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:43:03.950314 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d58e20c6-e9f1-4bcd-9452-58b27a4494e1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:03.951537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d58e20c6-e9f1-4bcd-9452-58b27a4494e1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:04.005526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d58e20c6-e9f1-4bcd-9452-58b27a4494e1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:04.012770 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d58e20c6-e9f1-4bcd-9452-58b27a4494e1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1258 microversion: 2.1 time: 0.066852 May 28 03:43:04.013391 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 458/946] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:03 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1258 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:05.030428 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-88dd8dfa-ca82-4966-9255-a7eb00231634 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:05.031140 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-88dd8dfa-ca82-4966-9255-a7eb00231634 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:05.038522 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-88dd8dfa-ca82-4966-9255-a7eb00231634 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:05.038950 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-88dd8dfa-ca82-4966-9255-a7eb00231634 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:05.039713 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-88dd8dfa-ca82-4966-9255-a7eb00231634 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:05.040151 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-88dd8dfa-ca82-4966-9255-a7eb00231634 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:05.104040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-88dd8dfa-ca82-4966-9255-a7eb00231634 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:05.110578 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-88dd8dfa-ca82-4966-9255-a7eb00231634 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1310 microversion: 2.1 time: 0.084058 May 28 03:43:05.111143 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 489/947] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:05 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1310 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:05.681483 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-dfcb2e43-18ef-4d68-adb3-42807a7bc087 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Action: 'create', calling method: .version_select at 0x7feb1feb0790>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-416578474", "metadata": {"Type": "work"}}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-dfcb2e43-18ef-4d68-adb3-42807a7bc087 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Request method failure captured: May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 121 May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZpJ3q_OPOhR0wUyPaCT-Ntgi-A3KSg_p8hBp5rijJ3kLkPcuONwydeSdSMw5rrEqio7iS9KFEqxfpCGThoWYAtQa-lqVrlA-Fe_XsCegftW5ZP1DTZUQH0kmClguPZp93u05F5odGMEFWA5NQJf11dSo9SkCMHFukOGhNmzk836G1k May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:43:05.687725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: c9eb085bc0014071b60c2deb59503134 May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-416578474", "metadata": {"Type": "work"}}} May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: May 28 03:43:05.689224 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-416578474', 'metadata': {'Type': 'work'}}}} May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise error May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi On instance['volume']: May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi '#$%' May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:05.690161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 28 03:43:05.691495 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 28 03:43:05.693090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 28 03:43:05.693090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.693090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-dfcb2e43-18ef-4d68-adb3-42807a7bc087 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:05.693090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-dfcb2e43-18ef-4d68-adb3-42807a7bc087 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] 10.222.0.41 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.138897 May 28 03:43:05.693090 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 459/948] 10.222.0.41 () {60 vars in 1155 bytes} [Sat May 28 03:43:05 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 140 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 28 03:43:05.699606 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-48fe698b-d3e6-4961-bc8e-2910a1a33f2f tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Action: 'create', calling method: .version_select at 0x7feb1f6f9ee0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-823919172", "metadata": {"Type": "work"}}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-48fe698b-d3e6-4961-bc8e-2910a1a33f2f tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Request method failure captured: May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 119 May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZpJ3q_OPOhR0wUyPaCT-Ntgi-A3KSg_p8hBp5rijJ3kLkPcuONwydeSdSMw5rrEqio7iS9KFEqxfpCGThoWYAtQa-lqVrlA-Fe_XsCegftW5ZP1DTZUQH0kmClguPZp93u05F5odGMEFWA5NQJf11dSo9SkCMHFukOGhNmzk836G1k May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:43:05.703184 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: c9eb085bc0014071b60c2deb59503134 May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-823919172", "metadata": {"Type": "work"}}} May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: May 28 03:43:05.707621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-823919172', 'metadata': {'Type': 'work'}}}} May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise error May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi On instance['volume']: May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi '0' May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:05.708429 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 28 03:43:05.709870 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 28 03:43:05.712046 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 28 03:43:05.712046 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.712046 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-48fe698b-d3e6-4961-bc8e-2910a1a33f2f tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:05.712046 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-48fe698b-d3e6-4961-bc8e-2910a1a33f2f tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] 10.222.0.41 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.008868 May 28 03:43:05.712046 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 490/949] 10.222.0.41 () {60 vars in 1155 bytes} [Sat May 28 03:43:05 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) May 28 03:43:05.714791 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d0fbfd20-1343-4343-95b1-772a319fcbed tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Action: 'create', calling method: .version_select at 0x7feb1fef99d0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1743730103", "metadata": {"Type": "work"}}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d0fbfd20-1343-4343-95b1-772a319fcbed tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Request method failure captured: May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 119 May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZpJ3q_OPOhR0wUyPaCT-Ntgi-A3KSg_p8hBp5rijJ3kLkPcuONwydeSdSMw5rrEqio7iS9KFEqxfpCGThoWYAtQa-lqVrlA-Fe_XsCegftW5ZP1DTZUQH0kmClguPZp93u05F5odGMEFWA5NQJf11dSo9SkCMHFukOGhNmzk836G1k May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:43:05.717494 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: c9eb085bc0014071b60c2deb59503134 May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1743730103", "metadata": {"Type": "work"}}} May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: May 28 03:43:05.720168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1743730103', 'metadata': {'Type': 'work'}}}} May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise error May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi On instance['volume']: May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi '' May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:05.721851 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 28 03:43:05.726343 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 28 03:43:05.729036 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 28 03:43:05.729036 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.729036 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d0fbfd20-1343-4343-95b1-772a319fcbed tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:05.729036 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d0fbfd20-1343-4343-95b1-772a319fcbed tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] 10.222.0.41 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.008035 May 28 03:43:05.729634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-be5e9eec-0967-437d-b18e-81b31a184ad7 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Calling method '.version_select at 0x7feb1f77ed30>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:05.733708 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 460/950] 10.222.0.41 () {60 vars in 1155 bytes} [Sat May 28 03:43:05 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 23 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-be5e9eec-0967-437d-b18e-81b31a184ad7 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] HTTP exception thrown: Volume tempest-invalid-640271579 could not be found. May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-be5e9eec-0967-437d-b18e-81b31a184ad7 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Request method failure captured: May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-640271579 HTTP/1.1 May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZpJ3q_OPOhR0wUyPaCT-Ntgi-A3KSg_p8hBp5rijJ3kLkPcuONwydeSdSMw5rrEqio7iS9KFEqxfpCGThoWYAtQa-lqVrlA-Fe_XsCegftW5ZP1DTZUQH0kmClguPZp93u05F5odGMEFWA5NQJf11dSo9SkCMHFukOGhNmzk836G1k May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:43:05.770502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: c9eb085bc0014071b60c2deb59503134 May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Volume tempest-invalid-640271579 could not be found. May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': 'tempest-invalid-640271579'} May 28 03:43:05.772210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-640271579 could not be found. (HTTP 404) (Request-ID: req-16983663-83d5-437a-9593-016283c63493) May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.773223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 28 03:43:05.774878 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-640271579 could not be found. May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:05.777301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-640271579 could not be found. May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-640271579 could not be found. May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-be5e9eec-0967-437d-b18e-81b31a184ad7 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Returning 404 to user: Volume tempest-invalid-640271579 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-be5e9eec-0967-437d-b18e-81b31a184ad7 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] 10.222.0.41 "DELETE /compute/v2.1/os-volumes/tempest-invalid-640271579" status: 404 len: 98 microversion: 2.1 time: 0.048012 May 28 03:43:05.780061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 491/951] 10.222.0.41 () {58 vars in 1214 bytes} [Sat May 28 03:43:05 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-640271579 => generated 98 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) May 28 03:43:05.782898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4fe00b88-4650-4888-ab23-0313eab4872b tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] 10.222.0.41 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003858 May 28 03:43:05.783603 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 461/952] 10.222.0.41 () {58 vars in 1139 bytes} [Sat May 28 03:43:05 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) May 28 03:43:05.792173 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-60b53a36-a33a-402d-b977-5f85f18fe9fb tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003836 May 28 03:43:05.792948 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 492/953] 10.222.0.41 () {58 vars in 1136 bytes} [Sat May 28 03:43:05 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) May 28 03:43:05.809938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-87820514-8bf9-4d16-ae97-9d00e1d118c9 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Calling method '.version_select at 0x7feb1feb0d30>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:05.839430 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-87820514-8bf9-4d16-ae97-9d00e1d118c9 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] HTTP exception thrown: Volume e168b776-d6ee-4410-8c93-8e0bbcb9e83c could not be found. May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-87820514-8bf9-4d16-ae97-9d00e1d118c9 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Request method failure captured: May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: DELETE /compute/v2.1/os-volumes/e168b776-d6ee-4410-8c93-8e0bbcb9e83c HTTP/1.1 May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZpJ3q_OPOhR0wUyPaCT-Ntgi-A3KSg_p8hBp5rijJ3kLkPcuONwydeSdSMw5rrEqio7iS9KFEqxfpCGThoWYAtQa-lqVrlA-Fe_XsCegftW5ZP1DTZUQH0kmClguPZp93u05F5odGMEFWA5NQJf11dSo9SkCMHFukOGhNmzk836G1k May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:43:05.841570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: c9eb085bc0014071b60c2deb59503134 May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Volume e168b776-d6ee-4410-8c93-8e0bbcb9e83c could not be found. May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': 'e168b776-d6ee-4410-8c93-8e0bbcb9e83c'} May 28 03:43:05.843005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e168b776-d6ee-4410-8c93-8e0bbcb9e83c could not be found. (HTTP 404) (Request-ID: req-51542ac3-e10e-4923-9d74-07b2a9d4a914) May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.843803 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 28 03:43:05.844936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e168b776-d6ee-4410-8c93-8e0bbcb9e83c could not be found. May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 28 03:43:05.846171 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e168b776-d6ee-4410-8c93-8e0bbcb9e83c could not be found. May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e168b776-d6ee-4410-8c93-8e0bbcb9e83c could not be found. May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-87820514-8bf9-4d16-ae97-9d00e1d118c9 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Returning 404 to user: Volume e168b776-d6ee-4410-8c93-8e0bbcb9e83c could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-87820514-8bf9-4d16-ae97-9d00e1d118c9 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] 10.222.0.41 "DELETE /compute/v2.1/os-volumes/e168b776-d6ee-4410-8c93-8e0bbcb9e83c" status: 404 len: 109 microversion: 2.1 time: 0.037998 May 28 03:43:05.847370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 462/954] 10.222.0.41 () {58 vars in 1247 bytes} [Sat May 28 03:43:05 2022] DELETE /compute/v2.1/os-volumes/e168b776-d6ee-4410-8c93-8e0bbcb9e83c => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:43:05.853031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-65289dd9-0e2d-4ed7-bbab-6705caca05d3 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Calling method '.version_select at 0x7feb1f5d70d0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:05.882039 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-65289dd9-0e2d-4ed7-bbab-6705caca05d3 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] HTTP exception thrown: Volume bd3712b7-4f65-4082-8b62-d0122cf98645 could not be found. May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-65289dd9-0e2d-4ed7-bbab-6705caca05d3 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Request method failure captured: May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: GET /compute/v2.1/os-volumes/bd3712b7-4f65-4082-8b62-d0122cf98645 HTTP/1.1 May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZpJ3q_OPOhR0wUyPaCT-Ntgi-A3KSg_p8hBp5rijJ3kLkPcuONwydeSdSMw5rrEqio7iS9KFEqxfpCGThoWYAtQa-lqVrlA-Fe_XsCegftW5ZP1DTZUQH0kmClguPZp93u05F5odGMEFWA5NQJf11dSo9SkCMHFukOGhNmzk836G1k May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:43:05.884434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_00a59e70e3764eb79d8d096d0307eeb2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 00a59e70e3764eb79d8d096d0307eeb2 May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-VolumesNegativeTest-1174040012 May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: c9eb085bc0014071b60c2deb59503134 May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-VolumesNegativeTest-1174040012-project May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Volume bd3712b7-4f65-4082-8b62-d0122cf98645 could not be found. May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': 'bd3712b7-4f65-4082-8b62-d0122cf98645'} May 28 03:43:05.892979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi item = cinderclient( May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume bd3712b7-4f65-4082-8b62-d0122cf98645 could not be found. (HTTP 404) (Request-ID: req-53b01b99-9442-4b14-b10a-fbcef9589ccd) May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.893831 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi item = cinderclient( May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 28 03:43:05.894939 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume bd3712b7-4f65-4082-8b62-d0122cf98645 could not be found. May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 28 03:43:05.896042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume bd3712b7-4f65-4082-8b62-d0122cf98645 could not be found. May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume bd3712b7-4f65-4082-8b62-d0122cf98645 could not be found. May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-65289dd9-0e2d-4ed7-bbab-6705caca05d3 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] Returning 404 to user: Volume bd3712b7-4f65-4082-8b62-d0122cf98645 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-65289dd9-0e2d-4ed7-bbab-6705caca05d3 tempest-VolumesNegativeTest-1174040012 tempest-VolumesNegativeTest-1174040012-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/bd3712b7-4f65-4082-8b62-d0122cf98645" status: 404 len: 109 microversion: 2.1 time: 0.037512 May 28 03:43:05.897396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 493/955] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:43:05 2022] GET /compute/v2.1/os-volumes/bd3712b7-4f65-4082-8b62-d0122cf98645 => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 28 03:43:06.135116 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-80ce52bb-307f-4663-b58d-d0bebdc733c7 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:06.135872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-80ce52bb-307f-4663-b58d-d0bebdc733c7 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:06.159450 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-80ce52bb-307f-4663-b58d-d0bebdc733c7 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:06.161800 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-80ce52bb-307f-4663-b58d-d0bebdc733c7 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:06.164823 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-80ce52bb-307f-4663-b58d-d0bebdc733c7 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:06.167201 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-80ce52bb-307f-4663-b58d-d0bebdc733c7 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:06.523655 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-80ce52bb-307f-4663-b58d-d0bebdc733c7 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:06.531223 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-80ce52bb-307f-4663-b58d-d0bebdc733c7 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.404009 May 28 03:43:06.541554 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 463/956] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:06 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:07.419149 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-8231a029-2797-40c9-a5a9-830852249157 req-2ad889e9-c298-4237-90a9-b7ba8f9b1684 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a799146f-de30-4095-8dfe-29d2b99837da", "tag": "88c75168-a0ab-4da4-b23a-190a9097ec15"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:07.457382 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-8231a029-2797-40c9-a5a9-830852249157 req-2ad889e9-c298-4237-90a9-b7ba8f9b1684 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:07.458605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-8231a029-2797-40c9-a5a9-830852249157 req-2ad889e9-c298-4237-90a9-b7ba8f9b1684 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:07.489579 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=82,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:07.494018 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-8231a029-2797-40c9-a5a9-830852249157 req-2ad889e9-c298-4237-90a9-b7ba8f9b1684 service nova] Creating event network-changed:88c75168-a0ab-4da4-b23a-190a9097ec15 for instance a799146f-de30-4095-8dfe-29d2b99837da on hv-cinder-84336 May 28 03:43:07.509166 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-8231a029-2797-40c9-a5a9-830852249157 req-2ad889e9-c298-4237-90a9-b7ba8f9b1684 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.103729 May 28 03:43:07.512445 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 494/957] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:43:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:07.552521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7806ceeb-0c38-42a5-9449-60d512098fe2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:07.553716 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7806ceeb-0c38-42a5-9449-60d512098fe2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:07.568861 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7806ceeb-0c38-42a5-9449-60d512098fe2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:07.569510 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7806ceeb-0c38-42a5-9449-60d512098fe2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:07.570192 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7806ceeb-0c38-42a5-9449-60d512098fe2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:07.570665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7806ceeb-0c38-42a5-9449-60d512098fe2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:07.814715 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7806ceeb-0c38-42a5-9449-60d512098fe2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:07.825073 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7806ceeb-0c38-42a5-9449-60d512098fe2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.278209 May 28 03:43:07.836605 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 464/958] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:07 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:08.855217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-09cad562-3a1e-46fc-bec3-5158bac29039 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:08.856175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-09cad562-3a1e-46fc-bec3-5158bac29039 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:08.864139 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09cad562-3a1e-46fc-bec3-5158bac29039 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:08.864916 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09cad562-3a1e-46fc-bec3-5158bac29039 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:08.866159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09cad562-3a1e-46fc-bec3-5158bac29039 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:08.866903 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09cad562-3a1e-46fc-bec3-5158bac29039 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:09.049247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-09cad562-3a1e-46fc-bec3-5158bac29039 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:09.058749 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-09cad562-3a1e-46fc-bec3-5158bac29039 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.208489 May 28 03:43:09.059319 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 495/959] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:08 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:10.083562 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d6e074a6-c848-4baa-bd33-f5a4beda1dee tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:10.085198 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d6e074a6-c848-4baa-bd33-f5a4beda1dee tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:10.102592 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d6e074a6-c848-4baa-bd33-f5a4beda1dee tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:10.106634 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d6e074a6-c848-4baa-bd33-f5a4beda1dee tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:10.109000 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d6e074a6-c848-4baa-bd33-f5a4beda1dee tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:10.118215 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d6e074a6-c848-4baa-bd33-f5a4beda1dee tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:10.288208 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d6e074a6-c848-4baa-bd33-f5a4beda1dee tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:10.296132 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d6e074a6-c848-4baa-bd33-f5a4beda1dee tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.223996 May 28 03:43:10.296730 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 465/960] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:10 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:11.314536 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8bef7af1-59c2-4878-addc-b22b8bdf4577 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:11.315247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8bef7af1-59c2-4878-addc-b22b8bdf4577 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:11.321949 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8bef7af1-59c2-4878-addc-b22b8bdf4577 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:11.322401 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8bef7af1-59c2-4878-addc-b22b8bdf4577 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:11.323172 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8bef7af1-59c2-4878-addc-b22b8bdf4577 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:11.323605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8bef7af1-59c2-4878-addc-b22b8bdf4577 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:11.468463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8bef7af1-59c2-4878-addc-b22b8bdf4577 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:11.477749 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8bef7af1-59c2-4878-addc-b22b8bdf4577 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.166985 May 28 03:43:11.478696 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 496/961] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:11 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:12.495731 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e7741a65-7e23-42cc-a341-f098f53a983e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:12.496541 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e7741a65-7e23-42cc-a341-f098f53a983e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:12.505575 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7741a65-7e23-42cc-a341-f098f53a983e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:12.506018 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7741a65-7e23-42cc-a341-f098f53a983e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:12.507391 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7741a65-7e23-42cc-a341-f098f53a983e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:12.507821 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7741a65-7e23-42cc-a341-f098f53a983e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:12.654263 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:12.660188 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e7741a65-7e23-42cc-a341-f098f53a983e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:12.669357 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e7741a65-7e23-42cc-a341-f098f53a983e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.178809 May 28 03:43:12.670231 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 466/962] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:12 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:13.700389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-cb1694e1-1797-4f66-927b-955e381ae5b4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:13.701005 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-cb1694e1-1797-4f66-927b-955e381ae5b4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:13.703491 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=6 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:13.708559 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb1694e1-1797-4f66-927b-955e381ae5b4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:13.709029 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb1694e1-1797-4f66-927b-955e381ae5b4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:13.722237 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb1694e1-1797-4f66-927b-955e381ae5b4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:13.722237 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb1694e1-1797-4f66-927b-955e381ae5b4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:13.886054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-cb1694e1-1797-4f66-927b-955e381ae5b4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:13.893526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-cb1694e1-1797-4f66-927b-955e381ae5b4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.208603 May 28 03:43:13.894276 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 497/963] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:13 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:14.911070 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c2a557d7-862f-4707-9d62-46abc4546acd tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:14.911662 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c2a557d7-862f-4707-9d62-46abc4546acd tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:14.914896 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=8 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:14.920673 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c2a557d7-862f-4707-9d62-46abc4546acd tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:14.921228 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c2a557d7-862f-4707-9d62-46abc4546acd tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:14.921940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c2a557d7-862f-4707-9d62-46abc4546acd tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:14.922381 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c2a557d7-862f-4707-9d62-46abc4546acd tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:15.053928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c2a557d7-862f-4707-9d62-46abc4546acd tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:15.061306 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c2a557d7-862f-4707-9d62-46abc4546acd tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.154220 May 28 03:43:15.061876 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 467/964] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:14 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:16.078724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5ad39758-1859-41ab-adaa-db8731e2f56d tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:16.079346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5ad39758-1859-41ab-adaa-db8731e2f56d tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:16.086321 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5ad39758-1859-41ab-adaa-db8731e2f56d tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:16.086838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5ad39758-1859-41ab-adaa-db8731e2f56d tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:16.087596 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5ad39758-1859-41ab-adaa-db8731e2f56d tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:16.088033 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5ad39758-1859-41ab-adaa-db8731e2f56d tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:16.264937 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5ad39758-1859-41ab-adaa-db8731e2f56d tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:16.272514 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5ad39758-1859-41ab-adaa-db8731e2f56d tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.197643 May 28 03:43:16.273081 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 498/965] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:16 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:17.290153 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6286e349-3a62-4475-a3a1-f4925e7039b8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:17.290752 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6286e349-3a62-4475-a3a1-f4925e7039b8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:17.299890 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6286e349-3a62-4475-a3a1-f4925e7039b8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:17.300654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6286e349-3a62-4475-a3a1-f4925e7039b8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:17.302120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6286e349-3a62-4475-a3a1-f4925e7039b8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:17.302874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6286e349-3a62-4475-a3a1-f4925e7039b8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:17.307771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:17.444541 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6286e349-3a62-4475-a3a1-f4925e7039b8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:17.452532 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6286e349-3a62-4475-a3a1-f4925e7039b8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.166827 May 28 03:43:17.456579 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 468/966] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:17 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:18.470578 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0a3d9d10-407f-4505-bd14-eade9d2917c9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:18.471756 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0a3d9d10-407f-4505-bd14-eade9d2917c9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:18.478405 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a3d9d10-407f-4505-bd14-eade9d2917c9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:18.478979 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a3d9d10-407f-4505-bd14-eade9d2917c9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:18.479902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a3d9d10-407f-4505-bd14-eade9d2917c9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:18.480346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a3d9d10-407f-4505-bd14-eade9d2917c9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:18.624607 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0a3d9d10-407f-4505-bd14-eade9d2917c9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:18.634660 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0a3d9d10-407f-4505-bd14-eade9d2917c9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.167816 May 28 03:43:18.644357 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 499/967] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:18 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:19.652127 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d70fd0e8-0620-4782-9934-88bbb563d983 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:19.652902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d70fd0e8-0620-4782-9934-88bbb563d983 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:19.659504 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d70fd0e8-0620-4782-9934-88bbb563d983 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:19.659871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d70fd0e8-0620-4782-9934-88bbb563d983 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:19.660738 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d70fd0e8-0620-4782-9934-88bbb563d983 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:19.661493 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d70fd0e8-0620-4782-9934-88bbb563d983 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:19.822175 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d70fd0e8-0620-4782-9934-88bbb563d983 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:19.831356 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d70fd0e8-0620-4782-9934-88bbb563d983 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.184100 May 28 03:43:19.841400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 469/968] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:19 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:20.848933 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-7fdebaf6-456d-4078-83ea-ea3f296058b2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:20.849393 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-7fdebaf6-456d-4078-83ea-ea3f296058b2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:20.852596 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=43,nova_api:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:20.857463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7fdebaf6-456d-4078-83ea-ea3f296058b2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:20.857904 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7fdebaf6-456d-4078-83ea-ea3f296058b2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:20.858834 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7fdebaf6-456d-4078-83ea-ea3f296058b2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:20.859270 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-7fdebaf6-456d-4078-83ea-ea3f296058b2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:20.990027 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-7fdebaf6-456d-4078-83ea-ea3f296058b2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:20.998098 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-7fdebaf6-456d-4078-83ea-ea3f296058b2 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.153255 May 28 03:43:20.998671 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 500/969] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:20 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:21.926094 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-a8ba7925-f968-4458-bdaa-7baaf416b85c req-c9977eed-a8a8-4cf2-9617-427926e39bd8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a799146f-de30-4095-8dfe-29d2b99837da", "name": "network-vif-plugged", "status": "completed", "tag": "88c75168-a0ab-4da4-b23a-190a9097ec15"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:21.935314 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-a8ba7925-f968-4458-bdaa-7baaf416b85c req-c9977eed-a8a8-4cf2-9617-427926e39bd8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:21.935756 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-a8ba7925-f968-4458-bdaa-7baaf416b85c req-c9977eed-a8a8-4cf2-9617-427926e39bd8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:21.949011 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-a8ba7925-f968-4458-bdaa-7baaf416b85c req-c9977eed-a8a8-4cf2-9617-427926e39bd8 service nova] Creating event network-vif-plugged:88c75168-a0ab-4da4-b23a-190a9097ec15 for instance a799146f-de30-4095-8dfe-29d2b99837da on hv-cinder-84336 May 28 03:43:21.954158 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-a8ba7925-f968-4458-bdaa-7baaf416b85c req-c9977eed-a8a8-4cf2-9617-427926e39bd8 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032136 May 28 03:43:21.954857 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 470/970] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:43:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:22.016105 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9baf2161-0014-4c3f-af45-aa67670538b5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:22.016806 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9baf2161-0014-4c3f-af45-aa67670538b5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:22.026970 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9baf2161-0014-4c3f-af45-aa67670538b5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:22.027579 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9baf2161-0014-4c3f-af45-aa67670538b5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:22.028557 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9baf2161-0014-4c3f-af45-aa67670538b5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:22.028989 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9baf2161-0014-4c3f-af45-aa67670538b5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:22.176450 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-9baf2161-0014-4c3f-af45-aa67670538b5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:22.183891 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9baf2161-0014-4c3f-af45-aa67670538b5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.172364 May 28 03:43:22.186021 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 501/971] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:22 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:23.201542 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-404df854-3d91-4263-9684-c0744c942c1b tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:23.202241 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-404df854-3d91-4263-9684-c0744c942c1b tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:23.204846 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=50,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:23.210581 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-404df854-3d91-4263-9684-c0744c942c1b tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:23.211031 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-404df854-3d91-4263-9684-c0744c942c1b tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:23.211801 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-404df854-3d91-4263-9684-c0744c942c1b tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:23.212270 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-404df854-3d91-4263-9684-c0744c942c1b tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:23.349777 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-404df854-3d91-4263-9684-c0744c942c1b tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:23.358015 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-404df854-3d91-4263-9684-c0744c942c1b tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1358 microversion: 2.1 time: 0.160661 May 28 03:43:23.358585 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 471/972] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:23 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1358 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:23.583703 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d4e4b805-6e2e-4a76-830e-9c01205d1192 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:23.584633 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d4e4b805-6e2e-4a76-830e-9c01205d1192 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:23.591421 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4e4b805-6e2e-4a76-830e-9c01205d1192 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:23.592123 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4e4b805-6e2e-4a76-830e-9c01205d1192 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:23.592937 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4e4b805-6e2e-4a76-830e-9c01205d1192 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:23.593406 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d4e4b805-6e2e-4a76-830e-9c01205d1192 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:23.625237 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d4e4b805-6e2e-4a76-830e-9c01205d1192 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.045406 May 28 03:43:23.625724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 502/973] 10.222.0.41 () {60 vars in 1342 bytes} [Sat May 28 03:43:23 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:43:23.691828 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-749dd7cd-fe87-46e2-8fbd-327b810ad653 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:23.692319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-749dd7cd-fe87-46e2-8fbd-327b810ad653 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:23.699013 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-749dd7cd-fe87-46e2-8fbd-327b810ad653 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:23.699455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-749dd7cd-fe87-46e2-8fbd-327b810ad653 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:23.700298 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-749dd7cd-fe87-46e2-8fbd-327b810ad653 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:23.700725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-749dd7cd-fe87-46e2-8fbd-327b810ad653 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:23.736570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-749dd7cd-fe87-46e2-8fbd-327b810ad653 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:43:24.068581 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-749dd7cd-fe87-46e2-8fbd-327b810ad653 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "DELETE /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 204 len: 0 microversion: 2.20 time: 0.380593 May 28 03:43:24.070017 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 472/974] 10.222.0.41 () {60 vars in 1279 bytes} [Sat May 28 03:43:23 2022] DELETE /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 28 03:43:24.084283 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1d6ad01e-01ff-4f16-9ab8-ca7415b166b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:24.085622 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1d6ad01e-01ff-4f16-9ab8-ca7415b166b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:24.091905 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d6ad01e-01ff-4f16-9ab8-ca7415b166b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:24.092537 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d6ad01e-01ff-4f16-9ab8-ca7415b166b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:24.093364 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d6ad01e-01ff-4f16-9ab8-ca7415b166b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:24.093972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1d6ad01e-01ff-4f16-9ab8-ca7415b166b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:24.376616 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d3766305-15ce-448e-b333-65cfe877bfdb tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:24.377486 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d3766305-15ce-448e-b333-65cfe877bfdb tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:24.384730 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3766305-15ce-448e-b333-65cfe877bfdb tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:24.387329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3766305-15ce-448e-b333-65cfe877bfdb tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:24.387329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3766305-15ce-448e-b333-65cfe877bfdb tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:24.387329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3766305-15ce-448e-b333-65cfe877bfdb tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:24.473936 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1d6ad01e-01ff-4f16-9ab8-ca7415b166b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:24.482481 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1d6ad01e-01ff-4f16-9ab8-ca7415b166b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:24.484427 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1d6ad01e-01ff-4f16-9ab8-ca7415b166b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:24.485186 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1d6ad01e-01ff-4f16-9ab8-ca7415b166b2 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.404724 May 28 03:43:24.489257 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 503/975] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:43:24 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:43:24.537867 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d3766305-15ce-448e-b333-65cfe877bfdb tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:24.545579 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d3766305-15ce-448e-b333-65cfe877bfdb tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1547 microversion: 2.1 time: 0.173279 May 28 03:43:24.546146 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 473/976] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:24 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1547 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:43:24.562222 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c6beed40-c574-4f4c-a611-a95f52351335 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:24.562827 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c6beed40-c574-4f4c-a611-a95f52351335 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:24.569589 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c6beed40-c574-4f4c-a611-a95f52351335 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:24.570032 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c6beed40-c574-4f4c-a611-a95f52351335 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:24.570786 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c6beed40-c574-4f4c-a611-a95f52351335 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:24.571218 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c6beed40-c574-4f4c-a611-a95f52351335 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:24.701413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c6beed40-c574-4f4c-a611-a95f52351335 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:24.710836 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c6beed40-c574-4f4c-a611-a95f52351335 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1547 microversion: 2.1 time: 0.152010 May 28 03:43:24.711935 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 504/977] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:24 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1547 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:24.731600 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e8cb66b4-4e90-46ec-b751-a86a10f03e05 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:24.737436 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e8cb66b4-4e90-46ec-b751-a86a10f03e05 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:24.737436 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e8cb66b4-4e90-46ec-b751-a86a10f03e05 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Searching by: {'deleted': False, 'project_id': '23f8194639b14942b36011698807b031'} {{(pid=124605) get_all /opt/stack/nova/nova/compute/api.py:2962}} May 28 03:43:24.748975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e8cb66b4-4e90-46ec-b751-a86a10f03e05 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:24.748975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e8cb66b4-4e90-46ec-b751-a86a10f03e05 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:24.748975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e8cb66b4-4e90-46ec-b751-a86a10f03e05 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:24.748975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e8cb66b4-4e90-46ec-b751-a86a10f03e05 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:24.837995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.multi_cell_list [None req-e8cb66b4-4e90-46ec-b751-a86a10f03e05 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=124605) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} May 28 03:43:24.938573 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e8cb66b4-4e90-46ec-b751-a86a10f03e05 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.211812 May 28 03:43:24.938912 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 474/978] 10.222.0.41 () {58 vars in 1124 bytes} [Sat May 28 03:43:24 2022] GET /compute/v2.1/servers => generated 342 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:24.949526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d48929ed-e2cf-42ac-934a-b1a741c2d550 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:24.950080 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d48929ed-e2cf-42ac-934a-b1a741c2d550 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:24.956108 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d48929ed-e2cf-42ac-934a-b1a741c2d550 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:24.956938 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d48929ed-e2cf-42ac-934a-b1a741c2d550 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:24.957354 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d48929ed-e2cf-42ac-934a-b1a741c2d550 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:24.957786 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d48929ed-e2cf-42ac-934a-b1a741c2d550 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:25.100526 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d48929ed-e2cf-42ac-934a-b1a741c2d550 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:25.107952 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d48929ed-e2cf-42ac-934a-b1a741c2d550 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1547 microversion: 2.1 time: 0.162569 May 28 03:43:25.108529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 505/979] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:24 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1547 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:25.504004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f1b5d4a6-7d8a-4da5-b289-24c650dc368d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:25.504613 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-f1b5d4a6-7d8a-4da5-b289-24c650dc368d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:25.511786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1b5d4a6-7d8a-4da5-b289-24c650dc368d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:25.512334 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1b5d4a6-7d8a-4da5-b289-24c650dc368d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:25.513202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1b5d4a6-7d8a-4da5-b289-24c650dc368d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:25.513639 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-f1b5d4a6-7d8a-4da5-b289-24c650dc368d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:26.046053 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f1b5d4a6-7d8a-4da5-b289-24c650dc368d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:26.057770 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f1b5d4a6-7d8a-4da5-b289-24c650dc368d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:26.060015 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-f1b5d4a6-7d8a-4da5-b289-24c650dc368d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:26.061485 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f1b5d4a6-7d8a-4da5-b289-24c650dc368d tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1724 microversion: 2.20 time: 0.561588 May 28 03:43:26.062480 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 475/980] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:43:25 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1724 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 28 03:43:26.777715 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-e30196e9-d1c1-42a5-a241-7f4bdc3cf7c6 req-bc0f71ae-c7fd-48f1-9545-5b6529462a8c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6be2aeaa-8d90-4e51-87d2-d02a8566ac1a", "name": "network-vif-deleted", "tag": "73f87b73-9d7c-43a7-9bd2-dc7fb5450112"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:26.784436 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-e30196e9-d1c1-42a5-a241-7f4bdc3cf7c6 req-bc0f71ae-c7fd-48f1-9545-5b6529462a8c service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:26.784882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-e30196e9-d1c1-42a5-a241-7f4bdc3cf7c6 req-bc0f71ae-c7fd-48f1-9545-5b6529462a8c service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:26.796234 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-e30196e9-d1c1-42a5-a241-7f4bdc3cf7c6 req-bc0f71ae-c7fd-48f1-9545-5b6529462a8c service nova] Creating event network-vif-deleted:73f87b73-9d7c-43a7-9bd2-dc7fb5450112 for instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a on hv-cinder-84336 May 28 03:43:26.801237 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-e30196e9-d1c1-42a5-a241-7f4bdc3cf7c6 req-bc0f71ae-c7fd-48f1-9545-5b6529462a8c service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028580 May 28 03:43:26.801850 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 506/981] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:43:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:27.085478 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9ef71ced-0dd3-40b3-b163-098639f6ab94 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "63109e9d-e9ef-419f-bb4f-e836b2812c0a"}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:27.086610 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9ef71ced-0dd3-40b3-b163-098639f6ab94 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:27.089844 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-20684975-b5c9-4ff3-b47c-4eb7a3bb8be0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:27.090626 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-20684975-b5c9-4ff3-b47c-4eb7a3bb8be0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:27.094574 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9ef71ced-0dd3-40b3-b163-098639f6ab94 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:27.094574 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9ef71ced-0dd3-40b3-b163-098639f6ab94 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:27.094751 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9ef71ced-0dd3-40b3-b163-098639f6ab94 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:27.095189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9ef71ced-0dd3-40b3-b163-098639f6ab94 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:27.098590 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-20684975-b5c9-4ff3-b47c-4eb7a3bb8be0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:27.099304 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-20684975-b5c9-4ff3-b47c-4eb7a3bb8be0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:27.100671 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-20684975-b5c9-4ff3-b47c-4eb7a3bb8be0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:27.101086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-20684975-b5c9-4ff3-b47c-4eb7a3bb8be0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:27.172707 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-20684975-b5c9-4ff3-b47c-4eb7a3bb8be0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:27.183030 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-20684975-b5c9-4ff3-b47c-4eb7a3bb8be0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:27.184565 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-20684975-b5c9-4ff3-b47c-4eb7a3bb8be0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:27.185929 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-20684975-b5c9-4ff3-b47c-4eb7a3bb8be0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1656 microversion: 2.20 time: 0.100994 May 28 03:43:27.186495 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 507/982] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:43:27 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1656 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:43:28.203966 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4a9828f3-11b2-417d-915c-e2f398a5b767 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:28.204656 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4a9828f3-11b2-417d-915c-e2f398a5b767 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:28.211129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a9828f3-11b2-417d-915c-e2f398a5b767 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:28.211576 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a9828f3-11b2-417d-915c-e2f398a5b767 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:28.212366 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a9828f3-11b2-417d-915c-e2f398a5b767 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:28.212799 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4a9828f3-11b2-417d-915c-e2f398a5b767 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:28.279964 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4a9828f3-11b2-417d-915c-e2f398a5b767 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:28.282619 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4a9828f3-11b2-417d-915c-e2f398a5b767 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:28.284238 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4a9828f3-11b2-417d-915c-e2f398a5b767 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:28.285440 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4a9828f3-11b2-417d-915c-e2f398a5b767 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1656 microversion: 2.20 time: 0.085288 May 28 03:43:28.285990 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 508/983] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:43:28 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1656 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:43:29.303812 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b6dd9458-7610-4070-a0fa-34f06b38fee0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:29.304453 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b6dd9458-7610-4070-a0fa-34f06b38fee0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:29.311665 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b6dd9458-7610-4070-a0fa-34f06b38fee0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:29.317943 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b6dd9458-7610-4070-a0fa-34f06b38fee0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:29.317943 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b6dd9458-7610-4070-a0fa-34f06b38fee0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:29.317943 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b6dd9458-7610-4070-a0fa-34f06b38fee0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:29.388776 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b6dd9458-7610-4070-a0fa-34f06b38fee0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:29.399177 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b6dd9458-7610-4070-a0fa-34f06b38fee0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:29.400793 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b6dd9458-7610-4070-a0fa-34f06b38fee0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e0eb0ab5585747d098bbb0b18770f4c5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3611c72fb0b8472da98ba9b0c0d86843', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:29.404040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b6dd9458-7610-4070-a0fa-34f06b38fee0 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 200 len: 1365 microversion: 2.20 time: 0.102991 May 28 03:43:29.404040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 509/984] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:43:29 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 1365 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 28 03:43:29.904908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9ef71ced-0dd3-40b3-b163-098639f6ab94 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "POST /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.821431 May 28 03:43:29.904908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 476/985] 10.222.0.41 () {60 vars in 1322 bytes} [Sat May 28 03:43:27 2022] POST /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/os-volume_attachments => generated 194 bytes in 2824 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:30.419534 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4fd32ae9-cf60-4933-87fd-8aa4c5ac39e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:30.420141 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4fd32ae9-cf60-4933-87fd-8aa4c5ac39e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:30.428902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4fd32ae9-cf60-4933-87fd-8aa4c5ac39e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:30.428902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4fd32ae9-cf60-4933-87fd-8aa4c5ac39e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:30.428902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4fd32ae9-cf60-4933-87fd-8aa4c5ac39e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:30.428902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4fd32ae9-cf60-4933-87fd-8aa4c5ac39e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:30.442010 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-4fd32ae9-cf60-4933-87fd-8aa4c5ac39e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] HTTP exception thrown: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4fd32ae9-cf60-4933-87fd-8aa4c5ac39e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Request method failure captured: May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a HTTP/1.1 May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZiXoa35A6jROnZ8zMQef32Be4oOIf-urjr-I_rz1MgM2_b3S2PmayB1X84xgl-OkycZsqj_AQ7q_z4FJL3oFFaOgR07hPjMPlbq-6NjLaC0eWnozGEBu56f-_apeayHLhoZJdjUAE9qXMfIOywazcSomdQ-kwUGKS-fFaJNB3natDQ May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Openstack-Nova-Api-Version: 2.20 May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:43:30.443717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: e0eb0ab5585747d098bbb0b18770f4c5 May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:30.445373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': '6be2aeaa-8d90-4e51-87d2-d02a8566ac1a'} May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:30.446180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:30.448752 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:30.452707 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4fd32ae9-cf60-4933-87fd-8aa4c5ac39e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Returning 404 to user: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:30.452707 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4fd32ae9-cf60-4933-87fd-8aa4c5ac39e4 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 404 len: 111 microversion: 2.20 time: 0.029442 May 28 03:43:30.452707 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 510/986] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:43:30 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 28 03:43:30.455599 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9159d484-bd47-4c7b-8f6b-5b6e1f2e7d49 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '.version_select at 0x7feb1feb05e0>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:30.547287 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9159d484-bd47-4c7b-8f6b-5b6e1f2e7d49 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-183206554" status: 204 len: 0 microversion: 2.20 time: 0.095914 May 28 03:43:30.547910 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 477/987] 10.222.0.41 () {60 vars in 1258 bytes} [Sat May 28 03:43:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-183206554 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 28 03:43:32.656894 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-51728532-c1b0-49e1-995f-39d7712e3566 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:32.657431 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-51728532-c1b0-49e1-995f-39d7712e3566 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:32.665463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-51728532-c1b0-49e1-995f-39d7712e3566 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:32.669138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-51728532-c1b0-49e1-995f-39d7712e3566 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:32.669138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-51728532-c1b0-49e1-995f-39d7712e3566 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:32.669138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-51728532-c1b0-49e1-995f-39d7712e3566 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:32.682487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-51728532-c1b0-49e1-995f-39d7712e3566 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] HTTP exception thrown: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-51728532-c1b0-49e1-995f-39d7712e3566 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Request method failure captured: May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a HTTP/1.1 May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZiXoa35A6jROnZ8zMQef32Be4oOIf-urjr-I_rz1MgM2_b3S2PmayB1X84xgl-OkycZsqj_AQ7q_z4FJL3oFFaOgR07hPjMPlbq-6NjLaC0eWnozGEBu56f-_apeayHLhoZJdjUAE9qXMfIOywazcSomdQ-kwUGKS-fFaJNB3natDQ May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Openstack-Nova-Api-Version: 2.20 May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:43:32.684224 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: e0eb0ab5585747d098bbb0b18770f4c5 May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:32.685766 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': '6be2aeaa-8d90-4e51-87d2-d02a8566ac1a'} May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:32.687193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:32.688422 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:32.696460 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-51728532-c1b0-49e1-995f-39d7712e3566 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Returning 404 to user: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:32.696460 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-51728532-c1b0-49e1-995f-39d7712e3566 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "DELETE /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 404 len: 111 microversion: 2.20 time: 0.032605 May 28 03:43:32.696460 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 511/988] 10.222.0.41 () {60 vars in 1279 bytes} [Sat May 28 03:43:32 2022] DELETE /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 28 03:43:32.697167 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-89d2aa44-ceb2-43e9-91f1-74acf19c03ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:32.697167 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-89d2aa44-ceb2-43e9-91f1-74acf19c03ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:32.702798 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-89d2aa44-ceb2-43e9-91f1-74acf19c03ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:32.703413 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-89d2aa44-ceb2-43e9-91f1-74acf19c03ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:32.704017 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-89d2aa44-ceb2-43e9-91f1-74acf19c03ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:32.704455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-89d2aa44-ceb2-43e9-91f1-74acf19c03ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:32.719725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-89d2aa44-ceb2-43e9-91f1-74acf19c03ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] HTTP exception thrown: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-89d2aa44-ceb2-43e9-91f1-74acf19c03ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Request method failure captured: May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a HTTP/1.1 May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZiXoa35A6jROnZ8zMQef32Be4oOIf-urjr-I_rz1MgM2_b3S2PmayB1X84xgl-OkycZsqj_AQ7q_z4FJL3oFFaOgR07hPjMPlbq-6NjLaC0eWnozGEBu56f-_apeayHLhoZJdjUAE9qXMfIOywazcSomdQ-kwUGKS-fFaJNB3natDQ May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Openstack-Nova-Api-Version: 2.20 May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:43:32.720200 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: e0eb0ab5585747d098bbb0b18770f4c5 May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:32.721711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '6be2aeaa-8d90-4e51-87d2-d02a8566ac1a'} May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:32.722975 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. May 28 03:43:32.724441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:32.727397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-89d2aa44-ceb2-43e9-91f1-74acf19c03ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Returning 404 to user: Instance 6be2aeaa-8d90-4e51-87d2-d02a8566ac1a could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:32.727397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-89d2aa44-ceb2-43e9-91f1-74acf19c03ad tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a" status: 404 len: 111 microversion: 2.20 time: 0.031055 May 28 03:43:32.727397 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 478/989] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:43:32 2022] GET /compute/v2.1/servers/6be2aeaa-8d90-4e51-87d2-d02a8566ac1a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 28 03:43:34.262723 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-07895036-cda0-4f73-be88-8c7db8e0c995 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:34.263217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-07895036-cda0-4f73-be88-8c7db8e0c995 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:34.269736 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07895036-cda0-4f73-be88-8c7db8e0c995 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:34.270185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07895036-cda0-4f73-be88-8c7db8e0c995 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:34.270977 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07895036-cda0-4f73-be88-8c7db8e0c995 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:34.271409 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-07895036-cda0-4f73-be88-8c7db8e0c995 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:34.285175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-07895036-cda0-4f73-be88-8c7db8e0c995 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] HTTP exception thrown: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-07895036-cda0-4f73-be88-8c7db8e0c995 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Request method failure captured: May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 HTTP/1.1 May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZiXoa35A6jROnZ8zMQef32Be4oOIf-urjr-I_rz1MgM2_b3S2PmayB1X84xgl-OkycZsqj_AQ7q_z4FJL3oFFaOgR07hPjMPlbq-6NjLaC0eWnozGEBu56f-_apeayHLhoZJdjUAE9qXMfIOywazcSomdQ-kwUGKS-fFaJNB3natDQ May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Openstack-Nova-Api-Version: 2.20 May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:43:34.286746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: e0eb0ab5585747d098bbb0b18770f4c5 May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:43:34.289060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': '35a9c645-590f-49e1-ab03-af4167f10547'} May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:34.290175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:43:34.291658 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:34.293267 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-07895036-cda0-4f73-be88-8c7db8e0c995 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Returning 404 to user: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:34.293267 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-07895036-cda0-4f73-be88-8c7db8e0c995 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "DELETE /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 404 len: 111 microversion: 2.20 time: 0.030005 May 28 03:43:34.293267 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 512/990] 10.222.0.41 () {60 vars in 1279 bytes} [Sat May 28 03:43:34 2022] DELETE /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 28 03:43:34.297402 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-921c4731-1bbc-4c4a-87d1-e40531a17511 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:34.298014 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-921c4731-1bbc-4c4a-87d1-e40531a17511 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] [instance: 35a9c645-590f-49e1-ab03-af4167f10547] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:34.304446 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-921c4731-1bbc-4c4a-87d1-e40531a17511 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:34.304920 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-921c4731-1bbc-4c4a-87d1-e40531a17511 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:34.305708 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-921c4731-1bbc-4c4a-87d1-e40531a17511 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:34.306142 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-921c4731-1bbc-4c4a-87d1-e40531a17511 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:34.319436 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-921c4731-1bbc-4c4a-87d1-e40531a17511 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] HTTP exception thrown: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-921c4731-1bbc-4c4a-87d1-e40531a17511 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Request method failure captured: May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 HTTP/1.1 May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZiXoa35A6jROnZ8zMQef32Be4oOIf-urjr-I_rz1MgM2_b3S2PmayB1X84xgl-OkycZsqj_AQ7q_z4FJL3oFFaOgR07hPjMPlbq-6NjLaC0eWnozGEBu56f-_apeayHLhoZJdjUAE9qXMfIOywazcSomdQ-kwUGKS-fFaJNB3natDQ May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Openstack-Nova-Api-Version: 2.20 May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:43:34.320953 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_3611c72fb0b8472da98ba9b0c0d86843"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 3611c72fb0b8472da98ba9b0c0d86843 May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-685474044 May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: e0eb0ab5585747d098bbb0b18770f4c5 May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-AttachVolumeShelveTestJSON-685474044-project May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:43:34.322696 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '35a9c645-590f-49e1-ab03-af4167f10547'} May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:34.323745 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. May 28 03:43:34.324928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:34.326141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-921c4731-1bbc-4c4a-87d1-e40531a17511 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] Returning 404 to user: Instance 35a9c645-590f-49e1-ab03-af4167f10547 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:34.326141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-921c4731-1bbc-4c4a-87d1-e40531a17511 tempest-AttachVolumeShelveTestJSON-685474044 tempest-AttachVolumeShelveTestJSON-685474044-project] 10.222.0.41 "GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547" status: 404 len: 111 microversion: 2.20 time: 0.028961 May 28 03:43:34.326141 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 479/991] 10.222.0.41 () {60 vars in 1276 bytes} [Sat May 28 03:43:34 2022] GET /compute/v2.1/servers/35a9c645-590f-49e1-ab03-af4167f10547 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 28 03:43:37.554206 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-068edf54-1051-4964-80cc-b7e58d11e1f9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:37.556091 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-068edf54-1051-4964-80cc-b7e58d11e1f9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:37.564528 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-068edf54-1051-4964-80cc-b7e58d11e1f9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:37.565266 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-068edf54-1051-4964-80cc-b7e58d11e1f9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:37.566355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-068edf54-1051-4964-80cc-b7e58d11e1f9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:37.566800 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-068edf54-1051-4964-80cc-b7e58d11e1f9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:37.711070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-068edf54-1051-4964-80cc-b7e58d11e1f9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:37.723301 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-068edf54-1051-4964-80cc-b7e58d11e1f9 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1593 microversion: 2.1 time: 0.174001 May 28 03:43:37.735635 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 513/992] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:37 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1593 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:40.548762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:40.824469 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-6a8d546f-43ff-4f3d-904d-21daea2aa982 req-aa58b351-92e9-44c8-afbc-9a0e54af417b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a799146f-de30-4095-8dfe-29d2b99837da", "tag": "88c75168-a0ab-4da4-b23a-190a9097ec15"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:40.840117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-6a8d546f-43ff-4f3d-904d-21daea2aa982 req-aa58b351-92e9-44c8-afbc-9a0e54af417b service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:40.840567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-6a8d546f-43ff-4f3d-904d-21daea2aa982 req-aa58b351-92e9-44c8-afbc-9a0e54af417b service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:40.886443 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-6a8d546f-43ff-4f3d-904d-21daea2aa982 req-aa58b351-92e9-44c8-afbc-9a0e54af417b service nova] Creating event network-changed:88c75168-a0ab-4da4-b23a-190a9097ec15 for instance a799146f-de30-4095-8dfe-29d2b99837da on hv-cinder-84336 May 28 03:43:40.911082 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-6a8d546f-43ff-4f3d-904d-21daea2aa982 req-aa58b351-92e9-44c8-afbc-9a0e54af417b service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.371183 May 28 03:43:40.924690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 480/993] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:43:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:41.447443 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b0ad2dd0-e72d-44b2-9fc0-65f2dbc9e523 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Action: 'create', calling method: .version_select at 0x7feb1f68cd30>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1784942105"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:42.142652 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b0ad2dd0-e72d-44b2-9fc0-65f2dbc9e523 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 0.835725 May 28 03:43:42.143275 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 514/994] 10.222.0.41 () {60 vars in 1154 bytes} [Sat May 28 03:43:41 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 837 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) May 28 03:43:42.155783 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cb62adf7-e456-488a-8bee-4d54d1cb48a4 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1fe4b310>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:42.204210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-cb62adf7-e456-488a-8bee-4d54d1cb48a4 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790" status: 200 len: 345 microversion: 2.1 time: 0.053505 May 28 03:43:42.204797 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 481/995] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:43:42 2022] GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790 => generated 345 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:43:43.216621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-70bcaa55-eb48-42a0-8f69-3ba5c93582c5 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb2c1a39d0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:43.265646 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-70bcaa55-eb48-42a0-8f69-3ba5c93582c5 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790" status: 200 len: 346 microversion: 2.1 time: 0.053169 May 28 03:43:43.266220 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 515/996] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:43:43 2022] GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790 => generated 346 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:43.278894 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a463045f-7c6d-4f08-a4fb-dcf0c207f3cb tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Action: 'create', calling method: .version_select at 0x7feb1fefb1f0>, body: {"snapshot": {"volume_id": "8b015a67-aa80-4d43-83fa-c900ea29d790", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-174442584"}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:43.842709 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a463045f-7c6d-4f08-a4fb-dcf0c207f3cb tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.568263 May 28 03:43:43.851289 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 482/997] 10.222.0.41 () {60 vars in 1161 bytes} [Sat May 28 03:43:43 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:43:43.852355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0c192202-92d5-4072-a93c-67b24d695b8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-990483197"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:43.852355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0c192202-92d5-4072-a93c-67b24d695b8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:43.853937 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0c192202-92d5-4072-a93c-67b24d695b8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:43.854548 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0c192202-92d5-4072-a93c-67b24d695b8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:43.866124 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0c192202-92d5-4072-a93c-67b24d695b8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:43.866591 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0c192202-92d5-4072-a93c-67b24d695b8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:43.872189 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ec626af8-bb49-4606-bef1-8636405a12a1 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1fdfa8b0>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:43.881853 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:43.917928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ec626af8-bb49-4606-bef1-8636405a12a1 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11" status: 200 len: 286 microversion: 2.1 time: 0.061109 May 28 03:43:43.918760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 483/998] 10.222.0.41 () {58 vars in 1250 bytes} [Sat May 28 03:43:43 2022] GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11 => generated 286 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:43.927690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-62806405-272c-4ce8-a41e-0023ecee113e tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1fe65ca0>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:43.961018 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-62806405-272c-4ce8-a41e-0023ecee113e tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.033160 May 28 03:43:43.961018 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 484/999] 10.222.0.41 () {58 vars in 1139 bytes} [Sat May 28 03:43:43 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:43:43.969232 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2b2c25c7-3138-453f-8826-0db50853e1c0 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1fdeaf70>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:44.002428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2b2c25c7-3138-453f-8826-0db50853e1c0 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11" status: 200 len: 286 microversion: 2.1 time: 0.036866 May 28 03:43:44.003121 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 485/1000] 10.222.0.41 () {58 vars in 1250 bytes} [Sat May 28 03:43:43 2022] GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11 => generated 286 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:43:44.017837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.network.security_group_api [None req-0c192202-92d5-4072-a93c-67b24d695b8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Adding security group 058a59c8-29c7-4c4b-bf28-a705fd027d0c to port 88c75168-a0ab-4da4-b23a-190a9097ec15 May 28 03:43:44.575034 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0c192202-92d5-4072-a93c-67b24d695b8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "POST /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/action" status: 202 len: 0 microversion: 2.1 time: 0.738988 May 28 03:43:44.577094 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 516/1001] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:43:43 2022] POST /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/action => generated 0 bytes in 742 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:43:44.584785 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-7394bf27-dcd0-4189-912a-ccbf92e4ff51 req-ee418333-8749-461c-97d4-1beb6b4d8b15 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a799146f-de30-4095-8dfe-29d2b99837da", "tag": "88c75168-a0ab-4da4-b23a-190a9097ec15"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:44.586223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2910a37c-a51c-45d9-9991-f4e4166d837a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:44.587026 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2910a37c-a51c-45d9-9991-f4e4166d837a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:44.594073 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2910a37c-a51c-45d9-9991-f4e4166d837a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:44.594772 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2910a37c-a51c-45d9-9991-f4e4166d837a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:44.595727 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2910a37c-a51c-45d9-9991-f4e4166d837a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:44.596329 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2910a37c-a51c-45d9-9991-f4e4166d837a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:44.604661 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-7394bf27-dcd0-4189-912a-ccbf92e4ff51 req-ee418333-8749-461c-97d4-1beb6b4d8b15 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:44.604661 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-7394bf27-dcd0-4189-912a-ccbf92e4ff51 req-ee418333-8749-461c-97d4-1beb6b4d8b15 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:44.625190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-7394bf27-dcd0-4189-912a-ccbf92e4ff51 req-ee418333-8749-461c-97d4-1beb6b4d8b15 service nova] Creating event network-changed:88c75168-a0ab-4da4-b23a-190a9097ec15 for instance a799146f-de30-4095-8dfe-29d2b99837da on hv-cinder-84336 May 28 03:43:44.630220 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-7394bf27-dcd0-4189-912a-ccbf92e4ff51 req-ee418333-8749-461c-97d4-1beb6b4d8b15 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050607 May 28 03:43:44.631393 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 486/1002] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:43:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:44.777638 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-2910a37c-a51c-45d9-9991-f4e4166d837a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:44.786742 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2910a37c-a51c-45d9-9991-f4e4166d837a tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1758 microversion: 2.1 time: 0.205550 May 28 03:43:44.788057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 517/1003] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:44 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1758 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:43:45.016376 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cb4d781d-58d9-4f7a-9ed9-0f009f426748 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1fef9af0>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:45.048521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-cb4d781d-58d9-4f7a-9ed9-0f009f426748 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11" status: 200 len: 286 microversion: 2.1 time: 0.036689 May 28 03:43:45.049225 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 487/1004] 10.222.0.41 () {58 vars in 1250 bytes} [Sat May 28 03:43:45 2022] GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11 => generated 286 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:46.060446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6a212f00-5429-43a1-bd6d-71bcac5e2c97 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1f5e8af0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:46.092251 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6a212f00-5429-43a1-bd6d-71bcac5e2c97 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11" status: 200 len: 286 microversion: 2.1 time: 0.035604 May 28 03:43:46.092848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 518/1005] 10.222.0.41 () {58 vars in 1250 bytes} [Sat May 28 03:43:46 2022] GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11 => generated 286 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:47.103797 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-41ad6462-661f-487d-b5ec-c8c9218167db tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1fdeaa60>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:47.137228 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-41ad6462-661f-487d-b5ec-c8c9218167db tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11" status: 200 len: 287 microversion: 2.1 time: 0.037240 May 28 03:43:47.143186 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 488/1006] 10.222.0.41 () {58 vars in 1250 bytes} [Sat May 28 03:43:47 2022] GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11 => generated 287 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:47.149771 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-63b5707a-9848-4298-beee-b713bf1308b7 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1f77eca0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:47.290178 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-63b5707a-9848-4298-beee-b713bf1308b7 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "DELETE /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11" status: 202 len: 0 microversion: 2.1 time: 0.144695 May 28 03:43:47.290739 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 519/1007] 10.222.0.41 () {58 vars in 1253 bytes} [Sat May 28 03:43:47 2022] DELETE /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11 => generated 0 bytes in 147 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:43:47.298621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1c9af968-66aa-4138-8cb4-0a4416ea14f5 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1fe85700>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:47.328183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1c9af968-66aa-4138-8cb4-0a4416ea14f5 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11" status: 200 len: 286 microversion: 2.1 time: 0.033250 May 28 03:43:47.328778 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 489/1008] 10.222.0.41 () {58 vars in 1250 bytes} [Sat May 28 03:43:47 2022] GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11 => generated 286 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:48.339893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b2284cec-3595-4ebd-ac54-0eb62c896e22 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1f7a3a60>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:48.365732 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-b2284cec-3595-4ebd-ac54-0eb62c896e22 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] HTTP exception thrown: Snapshot 88e41743-c44b-49a7-a629-2e5373258c11 could not be found. May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b2284cec-3595-4ebd-ac54-0eb62c896e22 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Request method failure captured: May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11 HTTP/1.1 May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZptz2xXZElRrN2gHgEBO4nMaD7ls495_4dBN9MJ-PZP9fbws30VQ6_tN_LoE4GBfwOYoR1a-B_9BHIG6XdXVO0wZgtrCn8i_w7mKpcRbGj_bBuA8hBydaVJVDfr-1QfPuZfPHveCzHh2aqH4IsgwVVt8VVdgwaVd5PNYlt6eTG9MgE May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 0bd351ab95cf4b3d92dd7de1135d7b4a May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:43:48.368193 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 0bd351ab95cf4b3d92dd7de1135d7b4a May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-VolumesSnapshotsTestJSON-530868166-project May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 581bf68d6942466b9143b5535e35e580 May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-VolumesSnapshotsTestJSON-530868166-project May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Snapshot 88e41743-c44b-49a7-a629-2e5373258c11 could not be found. May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': '88e41743-c44b-49a7-a629-2e5373258c11'} May 28 03:43:48.375972 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 88e41743-c44b-49a7-a629-2e5373258c11 could not be found. (HTTP 404) (Request-ID: req-eccd9692-9ce1-42d8-aae3-b45e51f25bcd) May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:48.377054 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 28 03:43:48.378545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 88e41743-c44b-49a7-a629-2e5373258c11 could not be found. May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show May 28 03:43:48.379955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 88e41743-c44b-49a7-a629-2e5373258c11 could not be found. May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 88e41743-c44b-49a7-a629-2e5373258c11 could not be found. May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b2284cec-3595-4ebd-ac54-0eb62c896e22 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Returning 404 to user: Snapshot 88e41743-c44b-49a7-a629-2e5373258c11 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:48.381497 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b2284cec-3595-4ebd-ac54-0eb62c896e22 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11" status: 404 len: 111 microversion: 2.1 time: 0.034585 May 28 03:43:48.382584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 520/1009] 10.222.0.41 () {58 vars in 1250 bytes} [Sat May 28 03:43:48 2022] GET /compute/v2.1/os-snapshots/88e41743-c44b-49a7-a629-2e5373258c11 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 28 03:43:48.382786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a4278512-bc10-45ca-a89b-6b34ae91764f tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1fef9ca0>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:48.659620 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a4278512-bc10-45ca-a89b-6b34ae91764f tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "DELETE /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790" status: 202 len: 0 microversion: 2.1 time: 0.284212 May 28 03:43:48.660191 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 490/1010] 10.222.0.41 () {58 vars in 1247 bytes} [Sat May 28 03:43:48 2022] DELETE /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790 => generated 0 bytes in 286 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:43:48.669036 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9ef267f5-4f95-489e-b015-5d2f53fa3b2b tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb2c1a3820>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:48.732018 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9ef267f5-4f95-489e-b015-5d2f53fa3b2b tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790" status: 200 len: 345 microversion: 2.1 time: 0.067408 May 28 03:43:48.732581 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 521/1011] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:43:48 2022] GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790 => generated 345 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:43:49.747353 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-404ce27a-82de-40db-8448-1b8c97566896 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1fe85ee0>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:49.777329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-404ce27a-82de-40db-8448-1b8c97566896 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] HTTP exception thrown: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-404ce27a-82de-40db-8448-1b8c97566896 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Request method failure captured: May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790 HTTP/1.1 May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZptz2xXZElRrN2gHgEBO4nMaD7ls495_4dBN9MJ-PZP9fbws30VQ6_tN_LoE4GBfwOYoR1a-B_9BHIG6XdXVO0wZgtrCn8i_w7mKpcRbGj_bBuA8hBydaVJVDfr-1QfPuZfPHveCzHh2aqH4IsgwVVt8VVdgwaVd5PNYlt6eTG9MgE May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 0bd351ab95cf4b3d92dd7de1135d7b4a May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:43:49.779884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 0bd351ab95cf4b3d92dd7de1135d7b4a May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-VolumesSnapshotsTestJSON-530868166-project May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 581bf68d6942466b9143b5535e35e580 May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-VolumesSnapshotsTestJSON-530868166-project May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '8b015a67-aa80-4d43-83fa-c900ea29d790'} May 28 03:43:49.787168 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi item = cinderclient( May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. (HTTP 404) (Request-ID: req-2c7cba22-ff9c-4de4-b625-cb462d03fffb) May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:49.788435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi item = cinderclient( May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 28 03:43:49.791129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 28 03:43:49.793162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-404ce27a-82de-40db-8448-1b8c97566896 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Returning 404 to user: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-404ce27a-82de-40db-8448-1b8c97566896 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790" status: 404 len: 109 microversion: 2.1 time: 0.040080 May 28 03:43:49.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 491/1012] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:43:49 2022] GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790 => generated 109 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:43:49.796657 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6e0c95b0-1be8-476d-8e1a-4df4dfa305ef tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1f77e670>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:49.824435 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-6e0c95b0-1be8-476d-8e1a-4df4dfa305ef tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] HTTP exception thrown: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6e0c95b0-1be8-476d-8e1a-4df4dfa305ef tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Request method failure captured: May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790 HTTP/1.1 May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZptz2xXZElRrN2gHgEBO4nMaD7ls495_4dBN9MJ-PZP9fbws30VQ6_tN_LoE4GBfwOYoR1a-B_9BHIG6XdXVO0wZgtrCn8i_w7mKpcRbGj_bBuA8hBydaVJVDfr-1QfPuZfPHveCzHh2aqH4IsgwVVt8VVdgwaVd5PNYlt6eTG9MgE May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 0bd351ab95cf4b3d92dd7de1135d7b4a May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:43:49.826404 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 0bd351ab95cf4b3d92dd7de1135d7b4a May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-VolumesSnapshotsTestJSON-530868166-project May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 581bf68d6942466b9143b5535e35e580 May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-VolumesSnapshotsTestJSON-530868166-project May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': '8b015a67-aa80-4d43-83fa-c900ea29d790'} May 28 03:43:49.827882 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. (HTTP 404) (Request-ID: req-d08c3f0c-cc88-49e1-99ad-7e033e2a2de0) May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:49.828775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 28 03:43:49.830101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 28 03:43:49.831346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6e0c95b0-1be8-476d-8e1a-4df4dfa305ef tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Returning 404 to user: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6e0c95b0-1be8-476d-8e1a-4df4dfa305ef tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "DELETE /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790" status: 404 len: 109 microversion: 2.1 time: 0.035738 May 28 03:43:49.832724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 522/1013] 10.222.0.41 () {58 vars in 1247 bytes} [Sat May 28 03:43:49 2022] DELETE /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:43:49.836522 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-34112146-0812-4448-8ee8-3fbfe2d8a802 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Calling method '.version_select at 0x7feb1fef9b80>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-34112146-0812-4448-8ee8-3fbfe2d8a802 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] HTTP exception thrown: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-34112146-0812-4448-8ee8-3fbfe2d8a802 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Request method failure captured: May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790 HTTP/1.1 May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZptz2xXZElRrN2gHgEBO4nMaD7ls495_4dBN9MJ-PZP9fbws30VQ6_tN_LoE4GBfwOYoR1a-B_9BHIG6XdXVO0wZgtrCn8i_w7mKpcRbGj_bBuA8hBydaVJVDfr-1QfPuZfPHveCzHh2aqH4IsgwVVt8VVdgwaVd5PNYlt6eTG9MgE May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 0bd351ab95cf4b3d92dd7de1135d7b4a May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:43:49.866468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_0bd351ab95cf4b3d92dd7de1135d7b4a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 0bd351ab95cf4b3d92dd7de1135d7b4a May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-530868166 May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-VolumesSnapshotsTestJSON-530868166-project May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 581bf68d6942466b9143b5535e35e580 May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-VolumesSnapshotsTestJSON-530868166-project May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '8b015a67-aa80-4d43-83fa-c900ea29d790'} May 28 03:43:49.868295 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi item = cinderclient( May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. (HTTP 404) (Request-ID: req-123eb487-7479-4a13-96d7-7a34fe24f1d9) May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:49.869281 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi item = cinderclient( May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 28 03:43:49.870938 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 28 03:43:49.872319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:43:49.873746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.873746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.873746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:43:49.873746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.873746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:43:49.873746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:43:49.873746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:43:49.873746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:43:49.873746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:43:49.873746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. May 28 03:43:49.873746 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:43:49.874910 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-34112146-0812-4448-8ee8-3fbfe2d8a802 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] Returning 404 to user: Volume 8b015a67-aa80-4d43-83fa-c900ea29d790 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:43:49.875775 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-34112146-0812-4448-8ee8-3fbfe2d8a802 tempest-VolumesSnapshotsTestJSON-530868166 tempest-VolumesSnapshotsTestJSON-530868166-project] 10.222.0.41 "GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790" status: 404 len: 109 microversion: 2.1 time: 0.043551 May 28 03:43:49.876320 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 492/1014] 10.222.0.41 () {58 vars in 1244 bytes} [Sat May 28 03:43:49 2022] GET /compute/v2.1/os-volumes/8b015a67-aa80-4d43-83fa-c900ea29d790 => generated 109 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:43:58.735880 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-695a6716-4c9c-4dbd-9bb6-9a39297543f0 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:43:58.736936 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-695a6716-4c9c-4dbd-9bb6-9a39297543f0 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:58.739431 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=31 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:58.740110 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=9 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:58.746746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-695a6716-4c9c-4dbd-9bb6-9a39297543f0 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:58.747201 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-695a6716-4c9c-4dbd-9bb6-9a39297543f0 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:58.747985 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-695a6716-4c9c-4dbd-9bb6-9a39297543f0 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:58.748425 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-695a6716-4c9c-4dbd-9bb6-9a39297543f0 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:58.964329 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-695a6716-4c9c-4dbd-9bb6-9a39297543f0 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "POST /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/action" status: 202 len: 0 microversion: 2.1 time: 0.236030 May 28 03:43:58.968291 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 523/1015] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:43:58 2022] POST /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/action => generated 0 bytes in 238 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:43:58.973336 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=27,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:58.973336 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=72,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:58.986995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-83ecafb3-8a8c-4179-bb74-50886306e70e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:43:58.987848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-83ecafb3-8a8c-4179-bb74-50886306e70e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:43:58.994170 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-83ecafb3-8a8c-4179-bb74-50886306e70e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:58.994619 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-83ecafb3-8a8c-4179-bb74-50886306e70e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:58.995662 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-83ecafb3-8a8c-4179-bb74-50886306e70e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:43:58.996101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-83ecafb3-8a8c-4179-bb74-50886306e70e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:43:59.290492 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-83ecafb3-8a8c-4179-bb74-50886306e70e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:43:59.301202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-83ecafb3-8a8c-4179-bb74-50886306e70e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1750 microversion: 2.1 time: 0.331501 May 28 03:43:59.302132 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 493/1016] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:43:58 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1750 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:00.321819 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-924fcef2-802f-43f7-94a7-a7416d2902a4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:00.322533 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-924fcef2-802f-43f7-94a7-a7416d2902a4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:00.329280 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-924fcef2-802f-43f7-94a7-a7416d2902a4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:00.329770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-924fcef2-802f-43f7-94a7-a7416d2902a4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:00.330551 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-924fcef2-802f-43f7-94a7-a7416d2902a4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:00.330994 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-924fcef2-802f-43f7-94a7-a7416d2902a4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:00.499066 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-924fcef2-802f-43f7-94a7-a7416d2902a4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:00.507300 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-924fcef2-802f-43f7-94a7-a7416d2902a4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1755 microversion: 2.1 time: 0.190468 May 28 03:44:00.508415 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 524/1017] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:00 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1755 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:01.526487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-205cdfd0-7909-44ae-bef7-fe84a9c41c6c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:01.527113 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-205cdfd0-7909-44ae-bef7-fe84a9c41c6c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:01.534987 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-205cdfd0-7909-44ae-bef7-fe84a9c41c6c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:01.535435 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-205cdfd0-7909-44ae-bef7-fe84a9c41c6c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:01.536203 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-205cdfd0-7909-44ae-bef7-fe84a9c41c6c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:01.536830 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-205cdfd0-7909-44ae-bef7-fe84a9c41c6c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:01.809673 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-205cdfd0-7909-44ae-bef7-fe84a9c41c6c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:01.817720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-205cdfd0-7909-44ae-bef7-fe84a9c41c6c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1755 microversion: 2.1 time: 0.296242 May 28 03:44:01.818293 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 494/1018] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:01 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1755 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:02.836607 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-42d1a47e-040a-4f49-81c7-d2709b537db5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:02.837797 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-42d1a47e-040a-4f49-81c7-d2709b537db5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:02.845556 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-42d1a47e-040a-4f49-81c7-d2709b537db5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:02.846010 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-42d1a47e-040a-4f49-81c7-d2709b537db5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:02.846781 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-42d1a47e-040a-4f49-81c7-d2709b537db5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:02.847819 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-42d1a47e-040a-4f49-81c7-d2709b537db5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:03.001941 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-42d1a47e-040a-4f49-81c7-d2709b537db5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:03.011252 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-42d1a47e-040a-4f49-81c7-d2709b537db5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1755 microversion: 2.1 time: 0.179414 May 28 03:44:03.020897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 525/1019] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:02 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1755 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:04.028284 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-26be6557-1877-4c68-97bf-c34b361419c8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:04.028823 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-26be6557-1877-4c68-97bf-c34b361419c8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:04.035997 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26be6557-1877-4c68-97bf-c34b361419c8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:04.036441 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26be6557-1877-4c68-97bf-c34b361419c8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:04.040212 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26be6557-1877-4c68-97bf-c34b361419c8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:04.040680 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26be6557-1877-4c68-97bf-c34b361419c8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:04.262664 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-26be6557-1877-4c68-97bf-c34b361419c8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:04.270884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-26be6557-1877-4c68-97bf-c34b361419c8 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1755 microversion: 2.1 time: 0.246537 May 28 03:44:04.271455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 495/1020] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:04 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1755 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:05.288548 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2badc0f5-7aaf-4216-8fc8-3c6a1a675b98 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:05.291139 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2badc0f5-7aaf-4216-8fc8-3c6a1a675b98 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:05.298411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2badc0f5-7aaf-4216-8fc8-3c6a1a675b98 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:05.298851 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2badc0f5-7aaf-4216-8fc8-3c6a1a675b98 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:05.299884 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2badc0f5-7aaf-4216-8fc8-3c6a1a675b98 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:05.300569 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2badc0f5-7aaf-4216-8fc8-3c6a1a675b98 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:05.449182 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-2badc0f5-7aaf-4216-8fc8-3c6a1a675b98 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:05.457268 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2badc0f5-7aaf-4216-8fc8-3c6a1a675b98 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1755 microversion: 2.1 time: 0.173330 May 28 03:44:05.457842 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 526/1021] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:05 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1755 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:06.474399 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e0ed646f-43a2-40e1-b2be-376ff0ed6c13 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:06.475005 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e0ed646f-43a2-40e1-b2be-376ff0ed6c13 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:06.481799 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e0ed646f-43a2-40e1-b2be-376ff0ed6c13 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:06.482227 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e0ed646f-43a2-40e1-b2be-376ff0ed6c13 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:06.483035 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e0ed646f-43a2-40e1-b2be-376ff0ed6c13 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:06.483464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e0ed646f-43a2-40e1-b2be-376ff0ed6c13 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:06.639701 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e0ed646f-43a2-40e1-b2be-376ff0ed6c13 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:06.647999 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e0ed646f-43a2-40e1-b2be-376ff0ed6c13 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1755 microversion: 2.1 time: 0.177441 May 28 03:44:06.648595 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 496/1022] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:06 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1755 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:07.664675 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-0a7dc990-62af-4d64-8d4b-3f43369bdcc4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:07.665405 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-0a7dc990-62af-4d64-8d4b-3f43369bdcc4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:07.671978 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a7dc990-62af-4d64-8d4b-3f43369bdcc4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:07.672532 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a7dc990-62af-4d64-8d4b-3f43369bdcc4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:07.673341 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a7dc990-62af-4d64-8d4b-3f43369bdcc4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:07.673817 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-0a7dc990-62af-4d64-8d4b-3f43369bdcc4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:07.822387 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-0a7dc990-62af-4d64-8d4b-3f43369bdcc4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:07.835111 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-0a7dc990-62af-4d64-8d4b-3f43369bdcc4 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1755 microversion: 2.1 time: 0.174261 May 28 03:44:07.835760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 527/1023] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:07 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1755 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:08.852525 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-6116c75c-4e21-476b-9d89-8ef68c4520c1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:08.853178 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-6116c75c-4e21-476b-9d89-8ef68c4520c1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:08.859118 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6116c75c-4e21-476b-9d89-8ef68c4520c1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:08.859563 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6116c75c-4e21-476b-9d89-8ef68c4520c1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:08.860312 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6116c75c-4e21-476b-9d89-8ef68c4520c1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:08.860737 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-6116c75c-4e21-476b-9d89-8ef68c4520c1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:09.008715 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-6116c75c-4e21-476b-9d89-8ef68c4520c1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:09.016574 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-6116c75c-4e21-476b-9d89-8ef68c4520c1 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1755 microversion: 2.1 time: 0.168054 May 28 03:44:09.017161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 497/1024] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:08 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1755 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:10.033212 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8396a99b-69ab-411b-bb76-7af2cb177e3e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:10.033967 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8396a99b-69ab-411b-bb76-7af2cb177e3e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:10.041002 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8396a99b-69ab-411b-bb76-7af2cb177e3e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:10.041498 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8396a99b-69ab-411b-bb76-7af2cb177e3e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:10.042293 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8396a99b-69ab-411b-bb76-7af2cb177e3e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:10.042739 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8396a99b-69ab-411b-bb76-7af2cb177e3e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:10.194661 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8396a99b-69ab-411b-bb76-7af2cb177e3e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:10.207897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8396a99b-69ab-411b-bb76-7af2cb177e3e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1755 microversion: 2.1 time: 0.178586 May 28 03:44:10.208764 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 528/1025] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:10 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1755 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:11.225673 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-37265872-5527-43e8-933d-9c5b3b0c9515 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:11.226770 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-37265872-5527-43e8-933d-9c5b3b0c9515 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:11.233084 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-37265872-5527-43e8-933d-9c5b3b0c9515 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:11.233560 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-37265872-5527-43e8-933d-9c5b3b0c9515 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:11.234318 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-37265872-5527-43e8-933d-9c5b3b0c9515 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:11.234752 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-37265872-5527-43e8-933d-9c5b3b0c9515 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:11.401753 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-37265872-5527-43e8-933d-9c5b3b0c9515 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:11.409872 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-37265872-5527-43e8-933d-9c5b3b0c9515 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1758 microversion: 2.1 time: 0.188347 May 28 03:44:11.412899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 498/1026] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:11 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1758 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:33.629048 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:44:33.629585 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Sat May 28 03:44:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:44:38.994250 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-625361337", "imageRef": "fce08ca8-7566-464d-83e6-e11a84d870aa", "flavorRef": "452"}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:44:39.000602 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=12 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:39.001308 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:39.062419 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] validate_networks() for None {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:44:39.472211 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:44:39.472826 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:44:39.473330 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:44:39.473761 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:44:39.474177 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:44:39.484200 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:39.484650 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:39.485490 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:39.485925 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:39.506324 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Getting quotas for project 282a34892e974900926a37a76690258c. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:44:39.510282 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Getting quotas for user 20b48a7b54a84c569a1ba39148dc5e70 and project 282a34892e974900926a37a76690258c. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:44:39.521343 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:44:39.524168 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='fce08ca8-7566-464d-83e6-e11a84d870aa',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:44:39.698474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-68e8c233-674d-413f-a33c-0437832dbef7 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.707348 May 28 03:44:39.699501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 529/1027] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:44:38 2022] POST /compute/v2.1/servers => generated 380 bytes in 712 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:44:39.719008 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1c2102ab-ccf9-405b-ba9d-b7f45a65fbd9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:39.720677 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1c2102ab-ccf9-405b-ba9d-b7f45a65fbd9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:39.723218 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=12 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:39.724156 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=36 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:39.796290 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1c2102ab-ccf9-405b-ba9d-b7f45a65fbd9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:39.803717 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1c2102ab-ccf9-405b-ba9d-b7f45a65fbd9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1214 microversion: 2.1 time: 0.095522 May 28 03:44:39.808634 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 499/1028] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:39 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1214 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:40.298558 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-571b28e2-8655-439b-ba23-8368f6a3b713 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:40.299629 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-571b28e2-8655-439b-ba23-8368f6a3b713 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:40.308608 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-571b28e2-8655-439b-ba23-8368f6a3b713 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:40.309065 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-571b28e2-8655-439b-ba23-8368f6a3b713 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:40.309932 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-571b28e2-8655-439b-ba23-8368f6a3b713 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:40.310366 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-571b28e2-8655-439b-ba23-8368f6a3b713 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:40.349616 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-571b28e2-8655-439b-ba23-8368f6a3b713 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Going to try to terminate instance {{(pid=124604) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:44:40.691263 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-571b28e2-8655-439b-ba23-8368f6a3b713 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "DELETE /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 204 len: 0 microversion: 2.1 time: 0.394478 May 28 03:44:40.691263 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 530/1029] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:44:40 2022] DELETE /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:44:40.697561 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a76ac0b1-c9c3-42a0-a440-7e3993ce7466 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:40.698484 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a76ac0b1-c9c3-42a0-a440-7e3993ce7466 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:40.707999 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a76ac0b1-c9c3-42a0-a440-7e3993ce7466 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:40.707999 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a76ac0b1-c9c3-42a0-a440-7e3993ce7466 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:40.707999 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a76ac0b1-c9c3-42a0-a440-7e3993ce7466 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:40.707999 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a76ac0b1-c9c3-42a0-a440-7e3993ce7466 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:40.823079 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-dab91128-cf9c-406b-ab5e-3ac019ea7f0c tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:40.824086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-dab91128-cf9c-406b-ab5e-3ac019ea7f0c tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:40.846901 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a76ac0b1-c9c3-42a0-a440-7e3993ce7466 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:40.855054 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a76ac0b1-c9c3-42a0-a440-7e3993ce7466 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1577 microversion: 2.1 time: 0.161304 May 28 03:44:40.858103 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 500/1030] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:40 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1577 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:40.894099 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-dab91128-cf9c-406b-ab5e-3ac019ea7f0c tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:40.901802 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-dab91128-cf9c-406b-ab5e-3ac019ea7f0c tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1214 microversion: 2.1 time: 0.083477 May 28 03:44:40.902363 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 531/1031] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:40 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1214 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:44:41.872208 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3a1951e3-29a3-40c7-b333-b15c37e7e78b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:41.874093 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3a1951e3-29a3-40c7-b333-b15c37e7e78b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:41.881918 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3a1951e3-29a3-40c7-b333-b15c37e7e78b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:41.882469 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3a1951e3-29a3-40c7-b333-b15c37e7e78b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:41.883360 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3a1951e3-29a3-40c7-b333-b15c37e7e78b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:41.883870 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3a1951e3-29a3-40c7-b333-b15c37e7e78b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:41.925837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-46e33153-b74d-40fa-9680-f5a25a085194 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:41.926464 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-46e33153-b74d-40fa-9680-f5a25a085194 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:41.934857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-46e33153-b74d-40fa-9680-f5a25a085194 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:41.935311 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-46e33153-b74d-40fa-9680-f5a25a085194 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:41.936254 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-46e33153-b74d-40fa-9680-f5a25a085194 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:41.936726 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-46e33153-b74d-40fa-9680-f5a25a085194 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:42.001484 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-46e33153-b74d-40fa-9680-f5a25a085194 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:42.009989 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-46e33153-b74d-40fa-9680-f5a25a085194 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1218 microversion: 2.1 time: 0.094315 May 28 03:44:42.010569 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 532/1032] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:41 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1218 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:42.083469 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3a1951e3-29a3-40c7-b333-b15c37e7e78b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:42.090426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3a1951e3-29a3-40c7-b333-b15c37e7e78b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1577 microversion: 2.1 time: 0.222321 May 28 03:44:42.109606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 501/1033] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:41 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1577 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:43.029254 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-02c4181f-d8b2-4f2d-9410-cfda0b292659 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:43.029921 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-02c4181f-d8b2-4f2d-9410-cfda0b292659 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:43.039405 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02c4181f-d8b2-4f2d-9410-cfda0b292659 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:43.040106 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02c4181f-d8b2-4f2d-9410-cfda0b292659 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:43.042231 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02c4181f-d8b2-4f2d-9410-cfda0b292659 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:43.043988 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-02c4181f-d8b2-4f2d-9410-cfda0b292659 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:43.108675 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a76d62f9-8e48-4408-b9cf-19cd53c1fd42 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:43.112948 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a76d62f9-8e48-4408-b9cf-19cd53c1fd42 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:43.122082 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a76d62f9-8e48-4408-b9cf-19cd53c1fd42 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:43.123039 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a76d62f9-8e48-4408-b9cf-19cd53c1fd42 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:43.124348 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a76d62f9-8e48-4408-b9cf-19cd53c1fd42 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:43.124946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a76d62f9-8e48-4408-b9cf-19cd53c1fd42 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:43.243384 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-02c4181f-d8b2-4f2d-9410-cfda0b292659 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:43.257784 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-02c4181f-d8b2-4f2d-9410-cfda0b292659 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1274 microversion: 2.1 time: 0.235487 May 28 03:44:43.258996 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 533/1034] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:43 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1274 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:43.633934 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a76d62f9-8e48-4408-b9cf-19cd53c1fd42 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:43.646880 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a76d62f9-8e48-4408-b9cf-19cd53c1fd42 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1577 microversion: 2.1 time: 0.542953 May 28 03:44:43.647454 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 502/1035] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:43 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1577 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:44:44.102919 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-367492b1-aeda-48a4-8f34-bbdf1028a04a req-577b7a55-dba1-4643-9a94-aa2e64f82b25 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b53491f6-a319-4950-ac9e-a9b0bd7a45e2", "name": "network-vif-deleted", "tag": "ce3985bf-d85c-4d8a-be10-2df6ad569608"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:44:44.119585 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-367492b1-aeda-48a4-8f34-bbdf1028a04a req-577b7a55-dba1-4643-9a94-aa2e64f82b25 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:44.120019 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-367492b1-aeda-48a4-8f34-bbdf1028a04a req-577b7a55-dba1-4643-9a94-aa2e64f82b25 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:44.161634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-367492b1-aeda-48a4-8f34-bbdf1028a04a req-577b7a55-dba1-4643-9a94-aa2e64f82b25 service nova] Creating event network-vif-deleted:ce3985bf-d85c-4d8a-be10-2df6ad569608 for instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 on hv-cinder-84336 May 28 03:44:44.173277 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-367492b1-aeda-48a4-8f34-bbdf1028a04a req-577b7a55-dba1-4643-9a94-aa2e64f82b25 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074578 May 28 03:44:44.181751 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 534/1036] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:44:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:44:44.207861 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0751cd0e-9f65-44a5-a485-52b74056baaa tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-990483197"}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:44:44.208396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0751cd0e-9f65-44a5-a485-52b74056baaa tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:44.228919 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0751cd0e-9f65-44a5-a485-52b74056baaa tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:44.234966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0751cd0e-9f65-44a5-a485-52b74056baaa tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:44.234966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0751cd0e-9f65-44a5-a485-52b74056baaa tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:44.234966 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0751cd0e-9f65-44a5-a485-52b74056baaa tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:44.275346 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b52574e0-21d2-48ff-a4ed-98a5854d74fa tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:44.276058 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b52574e0-21d2-48ff-a4ed-98a5854d74fa tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:44.318070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b52574e0-21d2-48ff-a4ed-98a5854d74fa tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:44.318070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b52574e0-21d2-48ff-a4ed-98a5854d74fa tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:44.318954 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b52574e0-21d2-48ff-a4ed-98a5854d74fa tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:44.319760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b52574e0-21d2-48ff-a4ed-98a5854d74fa tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:44.686468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.network.security_group_api [None req-0751cd0e-9f65-44a5-a485-52b74056baaa tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Removing security group 058a59c8-29c7-4c4b-bf28-a705fd027d0c from port 88c75168-a0ab-4da4-b23a-190a9097ec15 May 28 03:44:44.730188 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b52574e0-21d2-48ff-a4ed-98a5854d74fa tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:44.757167 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b52574e0-21d2-48ff-a4ed-98a5854d74fa tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1272 microversion: 2.1 time: 0.483631 May 28 03:44:44.758982 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 535/1037] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:44 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1272 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:44:44.780487 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c9757300-9086-4dcd-8a1a-37aa38d5fc86 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:44.782748 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c9757300-9086-4dcd-8a1a-37aa38d5fc86 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:44.794675 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c9757300-9086-4dcd-8a1a-37aa38d5fc86 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:44.797812 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c9757300-9086-4dcd-8a1a-37aa38d5fc86 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:44.799203 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c9757300-9086-4dcd-8a1a-37aa38d5fc86 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:44.805262 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c9757300-9086-4dcd-8a1a-37aa38d5fc86 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:44.915524 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-c9757300-9086-4dcd-8a1a-37aa38d5fc86 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:44.926530 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c9757300-9086-4dcd-8a1a-37aa38d5fc86 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 200 len: 1226 microversion: 2.1 time: 0.166241 May 28 03:44:44.928441 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 536/1038] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:44 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 1226 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:45.501161 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0751cd0e-9f65-44a5-a485-52b74056baaa tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "POST /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/action" status: 202 len: 0 microversion: 2.1 time: 1.296867 May 28 03:44:45.503264 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 503/1039] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:44:44 2022] POST /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/action => generated 0 bytes in 1299 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:44:45.786328 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-95dd4e15-f69e-4563-a57d-a5af9052c6cb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:45.787559 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-95dd4e15-f69e-4563-a57d-a5af9052c6cb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:45.795210 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-95dd4e15-f69e-4563-a57d-a5af9052c6cb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:45.795987 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-95dd4e15-f69e-4563-a57d-a5af9052c6cb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:45.796955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-95dd4e15-f69e-4563-a57d-a5af9052c6cb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:45.797584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-95dd4e15-f69e-4563-a57d-a5af9052c6cb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:45.944113 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-900879ef-0dae-4168-9c9b-ad21901ea974 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:45.945362 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-900879ef-0dae-4168-9c9b-ad21901ea974 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:45.954568 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-900879ef-0dae-4168-9c9b-ad21901ea974 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:45.959666 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-900879ef-0dae-4168-9c9b-ad21901ea974 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:45.960917 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-900879ef-0dae-4168-9c9b-ad21901ea974 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:45.961666 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-900879ef-0dae-4168-9c9b-ad21901ea974 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:45.993766 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-900879ef-0dae-4168-9c9b-ad21901ea974 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] HTTP exception thrown: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-900879ef-0dae-4168-9c9b-ad21901ea974 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Request method failure captured: May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 HTTP/1.1 May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZn0Qui27PFWt1C83-xg4yFPdjhPZnvWyBxCM-f8Qe2hd8rHuY3C9k6tR-bfxeJKyVsUF3_ynOM4ajKaj8L8A-8_VTS3F3IE_MTEvHpEo5Ag53NhTWKnyeMIUf8WBxjraePfkaBREJ6nRfxcVNjrDHcZ3Vo4xiTG2J-BAt0jGB4RLTo May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: c0fe831f6e954e05b7d448f1d4f43754 May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-TestVolumeBootPattern-1665013407 May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:44:46.001726 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-TestVolumeBootPattern-1665013407 May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: c0fe831f6e954e05b7d448f1d4f43754 May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-TestVolumeBootPattern-1665013407 May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-TestVolumeBootPattern-1665013407-project May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 2b515cebf18643c38f1cfc2e3fad1db4 May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-TestVolumeBootPattern-1665013407-project May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:44:46.003567 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': 'b53491f6-a319-4950-ac9e-a9b0bd7a45e2'} May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:44:46.004596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:44:46.006400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:44:46.010372 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-900879ef-0dae-4168-9c9b-ad21901ea974 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Returning 404 to user: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:44:46.014107 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-900879ef-0dae-4168-9c9b-ad21901ea974 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 404 len: 111 microversion: 2.1 time: 0.071530 May 28 03:44:46.014107 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 504/1040] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:45 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 111 bytes in 74 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:44:46.126288 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-95dd4e15-f69e-4563-a57d-a5af9052c6cb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:46.137804 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-95dd4e15-f69e-4563-a57d-a5af9052c6cb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1314 microversion: 2.1 time: 0.359968 May 28 03:44:46.140357 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 537/1041] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:45 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1314 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:46.185354 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-844783312", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1597361276", "security_groups": [{"name": "tempest-secgroup-smoke-1971281779"}], "block_device_mapping_v2": [{"uuid": "4b793144-3494-4b23-a6de-e09d47fb7c69", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "af6636fe-9088-4d20-9a1e-fa3df2392948"}]}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:44:46.209010 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-f852210c-9097-4ba4-99f2-7f22429e7f39 req-3d700fb7-6419-46c6-b198-7cf75b291006 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a799146f-de30-4095-8dfe-29d2b99837da", "tag": "88c75168-a0ab-4da4-b23a-190a9097ec15"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:44:46.226650 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-f852210c-9097-4ba4-99f2-7f22429e7f39 req-3d700fb7-6419-46c6-b198-7cf75b291006 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:46.227556 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-f852210c-9097-4ba4-99f2-7f22429e7f39 req-3d700fb7-6419-46c6-b198-7cf75b291006 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:46.242402 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-f852210c-9097-4ba4-99f2-7f22429e7f39 req-3d700fb7-6419-46c6-b198-7cf75b291006 service nova] Creating event network-changed:88c75168-a0ab-4da4-b23a-190a9097ec15 for instance a799146f-de30-4095-8dfe-29d2b99837da on hv-cinder-84336 May 28 03:44:46.250292 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-f852210c-9097-4ba4-99f2-7f22429e7f39 req-3d700fb7-6419-46c6-b198-7cf75b291006 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044805 May 28 03:44:46.252414 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 538/1042] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:44:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:44:46.998452 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.network.neutron [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] validate_networks() for [('af6636fe-9088-4d20-9a1e-fa3df2392948', None, None, None, None, None)] {{(pid=124605) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:44:47.155594 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d438c62e-d6d9-4454-a7c3-9c2bc63cbd6e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:47.156321 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d438c62e-d6d9-4454-a7c3-9c2bc63cbd6e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:47.162635 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d438c62e-d6d9-4454-a7c3-9c2bc63cbd6e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:47.168897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d438c62e-d6d9-4454-a7c3-9c2bc63cbd6e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:47.168897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d438c62e-d6d9-4454-a7c3-9c2bc63cbd6e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:47.168897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d438c62e-d6d9-4454-a7c3-9c2bc63cbd6e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:47.381825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d438c62e-d6d9-4454-a7c3-9c2bc63cbd6e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:47.392239 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d438c62e-d6d9-4454-a7c3-9c2bc63cbd6e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1314 microversion: 2.1 time: 0.241225 May 28 03:44:47.402335 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 539/1043] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:47 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1314 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:44:47.541759 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d3e9cd63-21d4-4266-a62f-1dd6cec2a9c5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:47.542269 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d3e9cd63-21d4-4266-a62f-1dd6cec2a9c5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:47.548453 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d3e9cd63-21d4-4266-a62f-1dd6cec2a9c5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:47.548835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d3e9cd63-21d4-4266-a62f-1dd6cec2a9c5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:47.551196 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d3e9cd63-21d4-4266-a62f-1dd6cec2a9c5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:47.551837 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d3e9cd63-21d4-4266-a62f-1dd6cec2a9c5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:47.653928 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-d3e9cd63-21d4-4266-a62f-1dd6cec2a9c5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lazy-loading 'services' on Instance uuid a799146f-de30-4095-8dfe-29d2b99837da {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:44:47.930719 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d3e9cd63-21d4-4266-a62f-1dd6cec2a9c5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "DELETE /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/os-volume_attachments/63109e9d-e9ef-419f-bb4f-e836b2812c0a" status: 202 len: 0 microversion: 2.1 time: 0.392711 May 28 03:44:47.936270 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 540/1044] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:44:47 2022] DELETE /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/os-volume_attachments/63109e9d-e9ef-419f-bb4f-e836b2812c0a => generated 0 bytes in 399 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 28 03:44:47.942863 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-3566f41d-52c4-41ad-98a1-f3a6c6602745 req-9e269ea8-05de-4d09-8089-8277d8b33530 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a799146f-de30-4095-8dfe-29d2b99837da", "tag": "88c75168-a0ab-4da4-b23a-190a9097ec15"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:44:47.954481 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-3566f41d-52c4-41ad-98a1-f3a6c6602745 req-9e269ea8-05de-4d09-8089-8277d8b33530 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:47.955290 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-3566f41d-52c4-41ad-98a1-f3a6c6602745 req-9e269ea8-05de-4d09-8089-8277d8b33530 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:47.972681 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-3566f41d-52c4-41ad-98a1-f3a6c6602745 req-9e269ea8-05de-4d09-8089-8277d8b33530 service nova] Creating event network-changed:88c75168-a0ab-4da4-b23a-190a9097ec15 for instance a799146f-de30-4095-8dfe-29d2b99837da on hv-cinder-84336 May 28 03:44:47.980048 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-3566f41d-52c4-41ad-98a1-f3a6c6602745 req-9e269ea8-05de-4d09-8089-8277d8b33530 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041610 May 28 03:44:47.981260 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 541/1045] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:44:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:44:48.002145 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3906891a-529d-42da-831b-b9ff1661b79e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:48.005744 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3906891a-529d-42da-831b-b9ff1661b79e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:48.014474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3906891a-529d-42da-831b-b9ff1661b79e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:48.015072 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3906891a-529d-42da-831b-b9ff1661b79e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:48.015927 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3906891a-529d-42da-831b-b9ff1661b79e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:48.016440 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3906891a-529d-42da-831b-b9ff1661b79e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:48.049073 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3906891a-529d-42da-831b-b9ff1661b79e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066252 May 28 03:44:48.049823 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 542/1046] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:44:47 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:44:48.311995 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Flavor limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:44:48.312426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Image limits 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:44:48.313126 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Flavor pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:44:48.313680 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Image pref 0:0:0 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:44:48.314844 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.virt.hardware [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:44:48.325602 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:48.326621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:48.327659 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:48.328302 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:48.386307 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Getting quotas for project c0fe831f6e954e05b7d448f1d4f43754. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:44:48.416743 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-813f744b-2c12-4a76-963c-df27e4fd7272 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:48.417502 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-813f744b-2c12-4a76-963c-df27e4fd7272 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:48.426432 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Getting quotas for user 2b515cebf18643c38f1cfc2e3fad1db4 and project c0fe831f6e954e05b7d448f1d4f43754. Resources: {'cores', 'instances', 'ram'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:44:48.439197 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-813f744b-2c12-4a76-963c-df27e4fd7272 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:48.439655 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-813f744b-2c12-4a76-963c-df27e4fd7272 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:48.440866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-813f744b-2c12-4a76-963c-df27e4fd7272 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:48.441551 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-813f744b-2c12-4a76-963c-df27e4fd7272 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:48.534329 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Going to run 1 instances... {{(pid=124605) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:44:48.536760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='4b793144-3494-4b23-a6de-e09d47fb7c69',volume_size=None,volume_type=None)] {{(pid=124605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:44:48.714230 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-813f744b-2c12-4a76-963c-df27e4fd7272 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:48.722565 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-813f744b-2c12-4a76-963c-df27e4fd7272 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1314 microversion: 2.1 time: 0.316699 May 28 03:44:48.723478 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 543/1047] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:48 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1314 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:44:48.728950 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-56308a70-8697-4afb-8a39-489754aeccc3 req-0d4d4239-0b62-4057-bd8e-705d0ae85b0e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "df122191-3ca2-4f3e-8076-99d5d5e713b0", "tag": "dd26d70b-fd3e-4adb-8cf1-11836f3ced7a"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:44:48.738100 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-56308a70-8697-4afb-8a39-489754aeccc3 req-0d4d4239-0b62-4057-bd8e-705d0ae85b0e service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:48.738408 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-56308a70-8697-4afb-8a39-489754aeccc3 req-0d4d4239-0b62-4057-bd8e-705d0ae85b0e service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:48.793806 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-56308a70-8697-4afb-8a39-489754aeccc3 req-0d4d4239-0b62-4057-bd8e-705d0ae85b0e service nova] Creating event network-changed:dd26d70b-fd3e-4adb-8cf1-11836f3ced7a for instance df122191-3ca2-4f3e-8076-99d5d5e713b0 on hv-cinder-84336 May 28 03:44:48.809183 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-56308a70-8697-4afb-8a39-489754aeccc3 req-0d4d4239-0b62-4057-bd8e-705d0ae85b0e service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083917 May 28 03:44:48.809799 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 544/1048] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:44:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:44:49.062634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-af790bd2-d6fd-4720-8f96-e20fac20ce8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:49.063641 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-af790bd2-d6fd-4720-8f96-e20fac20ce8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:49.075860 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af790bd2-d6fd-4720-8f96-e20fac20ce8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:49.076516 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af790bd2-d6fd-4720-8f96-e20fac20ce8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:49.077826 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af790bd2-d6fd-4720-8f96-e20fac20ce8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:49.079040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-af790bd2-d6fd-4720-8f96-e20fac20ce8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:49.123956 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d6b85788-ddd8-457b-820b-ecd2da17a759 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 2.945165 May 28 03:44:49.129487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 505/1049] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:44:46 2022] POST /compute/v2.1/servers => generated 406 bytes in 2951 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:44:49.132965 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-af790bd2-d6fd-4720-8f96-e20fac20ce8c tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.074883 May 28 03:44:49.133876 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 545/1050] 10.222.0.41 () {58 vars in 1301 bytes} [Sat May 28 03:44:49 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/os-volume_attachments => generated 25 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 28 03:44:49.142468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-fafc491b-3437-486d-b201-b155758d8647 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:49.143334 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-fafc491b-3437-486d-b201-b155758d8647 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:49.155940 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8b1daf02-960d-46eb-ac26-118a06ce8fea tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:49.161398 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8b1daf02-960d-46eb-ac26-118a06ce8fea tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:49.186518 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8b1daf02-960d-46eb-ac26-118a06ce8fea tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:49.187082 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8b1daf02-960d-46eb-ac26-118a06ce8fea tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:49.188026 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8b1daf02-960d-46eb-ac26-118a06ce8fea tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:49.188566 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8b1daf02-960d-46eb-ac26-118a06ce8fea tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:49.279510 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-fafc491b-3437-486d-b201-b155758d8647 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:49.292426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-fafc491b-3437-486d-b201-b155758d8647 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1094 microversion: 2.1 time: 0.160351 May 28 03:44:49.294428 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 506/1051] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:49 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1094 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:49.320756 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-8b1daf02-960d-46eb-ac26-118a06ce8fea tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] HTTP exception thrown: Instance a799146f-de30-4095-8dfe-29d2b99837da is not attached to volume 63109e9d-e9ef-419f-bb4f-e836b2812c0a May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8b1daf02-960d-46eb-ac26-118a06ce8fea tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Request method failure captured: May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/os-volume_attachments/63109e9d-e9ef-419f-bb4f-e836b2812c0a HTTP/1.1 May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZoxitRvHhrJeKeY-N2wgjumvGOZ5QlT8tA4Ny_zszUWSwzflpU1pG7cvrqv1qx56aTTz3CFK6Qn8toCs-3JjstETBtRUIUq7kE4jrGPlFnIzIJmoHSjzyDs7EcVNCw6QVT-xbdP0E5YM_AzwEZhTQXODVIHxOKA5OgVlER2W5zT9Eg May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 23f8194639b14942b36011698807b031 May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-TestMinimumBasicScenario-681911436 May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:44:49.323232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:44:49.330060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/23f8194639b14942b36011698807b031"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/23f8194639b14942b36011698807b031"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/23f8194639b14942b36011698807b031"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_23f8194639b14942b36011698807b031"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:44:49.330060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-TestMinimumBasicScenario-681911436 May 28 03:44:49.330060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 23f8194639b14942b36011698807b031 May 28 03:44:49.330060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-TestMinimumBasicScenario-681911436 May 28 03:44:49.330060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-TestMinimumBasicScenario-681911436-project May 28 03:44:49.330060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:44:49.330060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:44:49.330060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: c1a9600bad844f60937aa35126cd1be4 May 28 03:44:49.330060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-TestMinimumBasicScenario-681911436-project May 28 03:44:49.330060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:44:49.330060 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance a799146f-de30-4095-8dfe-29d2b99837da is not attached to volume 63109e9d-e9ef-419f-bb4f-e836b2812c0a May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'server_id': 'a799146f-de30-4095-8dfe-29d2b99837da', 'id': '63109e9d-e9ef-419f-bb4f-e836b2812c0a'} May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 63109e9d-e9ef-419f-bb4f-e836b2812c0a. May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:44:49.330835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a799146f-de30-4095-8dfe-29d2b99837da is not attached to volume 63109e9d-e9ef-419f-bb4f-e836b2812c0a May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a799146f-de30-4095-8dfe-29d2b99837da is not attached to volume 63109e9d-e9ef-419f-bb4f-e836b2812c0a May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:44:49.332138 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8b1daf02-960d-46eb-ac26-118a06ce8fea tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Returning 404 to user: Instance a799146f-de30-4095-8dfe-29d2b99837da is not attached to volume 63109e9d-e9ef-419f-bb4f-e836b2812c0a {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:44:49.334313 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8b1daf02-960d-46eb-ac26-118a06ce8fea tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "DELETE /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/os-volume_attachments/63109e9d-e9ef-419f-bb4f-e836b2812c0a" status: 404 len: 154 microversion: 2.1 time: 0.184486 May 28 03:44:49.334313 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 546/1052] 10.222.0.41 () {58 vars in 1415 bytes} [Sat May 28 03:44:49 2022] DELETE /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da/os-volume_attachments/63109e9d-e9ef-419f-bb4f-e836b2812c0a => generated 154 bytes in 189 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:44:49.744775 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4b6e9ef4-c124-4159-9b3e-8b1c5cef0a4f tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:49.745515 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4b6e9ef4-c124-4159-9b3e-8b1c5cef0a4f tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:49.764348 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4b6e9ef4-c124-4159-9b3e-8b1c5cef0a4f tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:49.765712 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4b6e9ef4-c124-4159-9b3e-8b1c5cef0a4f tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:49.766155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4b6e9ef4-c124-4159-9b3e-8b1c5cef0a4f tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:49.766625 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4b6e9ef4-c124-4159-9b3e-8b1c5cef0a4f tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:50.110951 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4b6e9ef4-c124-4159-9b3e-8b1c5cef0a4f tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:50.119715 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4b6e9ef4-c124-4159-9b3e-8b1c5cef0a4f tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1314 microversion: 2.1 time: 0.383461 May 28 03:44:50.120416 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 507/1053] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:49 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1314 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:50.311789 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-2c625fd1-a002-46e8-b492-86824321b1e1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:50.312533 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-2c625fd1-a002-46e8-b492-86824321b1e1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:50.315268 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=2 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:50.322292 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2c625fd1-a002-46e8-b492-86824321b1e1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:50.323133 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2c625fd1-a002-46e8-b492-86824321b1e1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:50.326987 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2c625fd1-a002-46e8-b492-86824321b1e1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:50.327790 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-2c625fd1-a002-46e8-b492-86824321b1e1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:50.357851 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:50.420613 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-2c625fd1-a002-46e8-b492-86824321b1e1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:50.429900 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-2c625fd1-a002-46e8-b492-86824321b1e1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1144 microversion: 2.1 time: 0.122020 May 28 03:44:50.431023 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 547/1054] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:50 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1144 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:50.704520 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-fab32e11-4275-4ee0-99e9-6c31310bce75 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:50.704520 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-fab32e11-4275-4ee0-99e9-6c31310bce75 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:50.708844 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fab32e11-4275-4ee0-99e9-6c31310bce75 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:50.709558 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fab32e11-4275-4ee0-99e9-6c31310bce75 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:50.710598 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fab32e11-4275-4ee0-99e9-6c31310bce75 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:50.711218 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-fab32e11-4275-4ee0-99e9-6c31310bce75 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:50.740020 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-fab32e11-4275-4ee0-99e9-6c31310bce75 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:44:51.092438 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-fab32e11-4275-4ee0-99e9-6c31310bce75 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "DELETE /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 204 len: 0 microversion: 2.1 time: 0.396627 May 28 03:44:51.099062 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 508/1055] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:44:50 2022] DELETE /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 0 bytes in 401 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:44:51.101922 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-75324ca1-078b-4531-b56f-1b231b47513f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:51.102540 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-75324ca1-078b-4531-b56f-1b231b47513f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:51.104741 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=8 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:51.110061 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-75324ca1-078b-4531-b56f-1b231b47513f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:51.110514 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-75324ca1-078b-4531-b56f-1b231b47513f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:51.111288 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-75324ca1-078b-4531-b56f-1b231b47513f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:51.111729 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-75324ca1-078b-4531-b56f-1b231b47513f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:51.273856 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-75324ca1-078b-4531-b56f-1b231b47513f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:51.281538 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-75324ca1-078b-4531-b56f-1b231b47513f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1553 microversion: 2.1 time: 0.183473 May 28 03:44:51.282535 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 548/1056] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:51 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1553 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:51.288652 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1ccc123a-1b90-4808-9135-b86b2ef2c44e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:51.289759 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1ccc123a-1b90-4808-9135-b86b2ef2c44e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:51.296389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1ccc123a-1b90-4808-9135-b86b2ef2c44e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:51.297394 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1ccc123a-1b90-4808-9135-b86b2ef2c44e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:51.298371 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1ccc123a-1b90-4808-9135-b86b2ef2c44e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:51.298977 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1ccc123a-1b90-4808-9135-b86b2ef2c44e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:51.314896 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-f6b0cd45-f531-4eb0-b7c1-7c501b1dd554 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Action: 'create', calling method: .version_select at 0x7feb1fe65e50>, body: {"keypair": {"name": "tempest-TestShelveInstance-1367140617", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL63ZoA9vASA9s+9bFuZZ4Iz8A3SNhjsfTP7wd+DNgiRDXC6FOSGwIwZS+dIDb0Y+MuFYe/+PJLa3zVFSnX+zpjg4DlcorkOC2pKylbM5VWRSjlnKeA0M8M9MOeW0ohk7Q=="}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:44:51.330016 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-f6b0cd45-f531-4eb0-b7c1-7c501b1dd554 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Getting quotas for project d1d9f51a3cc44062af827a498842ee80. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:44:51.334190 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.quota [None req-f6b0cd45-f531-4eb0-b7c1-7c501b1dd554 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Getting quotas for user 303285e036734e5cbd491920ae7473d9 and project d1d9f51a3cc44062af827a498842ee80. Resources: {'key_pairs'} {{(pid=124605) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:44:51.436384 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1ccc123a-1b90-4808-9135-b86b2ef2c44e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:51.444266 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1ccc123a-1b90-4808-9135-b86b2ef2c44e tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1314 microversion: 2.1 time: 0.160667 May 28 03:44:51.445495 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 549/1057] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:51 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:51.449922 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8f12ea32-4155-4a4d-a2ea-16828d14df29 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:51.450527 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8f12ea32-4155-4a4d-a2ea-16828d14df29 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:51.455779 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-f6b0cd45-f531-4eb0-b7c1-7c501b1dd554 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.329169 May 28 03:44:51.456689 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 509/1058] 10.222.0.41 () {60 vars in 1158 bytes} [Sat May 28 03:44:51 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:44:51.461213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8f12ea32-4155-4a4d-a2ea-16828d14df29 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:51.461674 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8f12ea32-4155-4a4d-a2ea-16828d14df29 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:51.464119 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8f12ea32-4155-4a4d-a2ea-16828d14df29 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:51.464119 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8f12ea32-4155-4a4d-a2ea-16828d14df29 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:51.533844 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8f12ea32-4155-4a4d-a2ea-16828d14df29 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:51.541827 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8f12ea32-4155-4a4d-a2ea-16828d14df29 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1144 microversion: 2.1 time: 0.095319 May 28 03:44:51.542437 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 550/1059] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:51 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1144 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:52.300040 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7cd6a954-876e-4ba1-8ebb-f282d995f0e5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:52.300932 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7cd6a954-876e-4ba1-8ebb-f282d995f0e5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:52.308762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7cd6a954-876e-4ba1-8ebb-f282d995f0e5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:52.309414 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7cd6a954-876e-4ba1-8ebb-f282d995f0e5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:52.310490 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7cd6a954-876e-4ba1-8ebb-f282d995f0e5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:52.311120 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7cd6a954-876e-4ba1-8ebb-f282d995f0e5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:52.462199 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3b3afa03-c810-4caf-b93f-22b4b716a2ce tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:52.464866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3b3afa03-c810-4caf-b93f-22b4b716a2ce tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:52.471434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3b3afa03-c810-4caf-b93f-22b4b716a2ce tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:52.472560 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3b3afa03-c810-4caf-b93f-22b4b716a2ce tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:52.473850 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3b3afa03-c810-4caf-b93f-22b4b716a2ce tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:52.474904 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3b3afa03-c810-4caf-b93f-22b4b716a2ce tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:52.668180 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3b3afa03-c810-4caf-b93f-22b4b716a2ce tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:52.677088 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3b3afa03-c810-4caf-b93f-22b4b716a2ce tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1314 microversion: 2.1 time: 0.220080 May 28 03:44:52.678891 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 551/1060] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:52 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1314 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:52.693677 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7cd6a954-876e-4ba1-8ebb-f282d995f0e5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:52.728029 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-784f051b-ec5d-4b50-b223-002ff4db5352 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:52.728860 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-784f051b-ec5d-4b50-b223-002ff4db5352 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:52.749211 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7cd6a954-876e-4ba1-8ebb-f282d995f0e5 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1553 microversion: 2.1 time: 0.452550 May 28 03:44:52.749730 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-784f051b-ec5d-4b50-b223-002ff4db5352 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:52.754281 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-784f051b-ec5d-4b50-b223-002ff4db5352 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:52.755120 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-784f051b-ec5d-4b50-b223-002ff4db5352 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:52.755567 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-784f051b-ec5d-4b50-b223-002ff4db5352 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:52.758743 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 510/1061] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:52 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1553 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:52.857639 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-784f051b-ec5d-4b50-b223-002ff4db5352 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:52.865876 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-784f051b-ec5d-4b50-b223-002ff4db5352 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1136 microversion: 2.1 time: 0.184549 May 28 03:44:52.866461 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 552/1062] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:52 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1136 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:53.694595 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7aa3ae2e-8f6b-44f7-b26a-8a5cfe83c06b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:53.695855 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7aa3ae2e-8f6b-44f7-b26a-8a5cfe83c06b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:53.708400 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7aa3ae2e-8f6b-44f7-b26a-8a5cfe83c06b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:53.708855 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7aa3ae2e-8f6b-44f7-b26a-8a5cfe83c06b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:53.712146 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7aa3ae2e-8f6b-44f7-b26a-8a5cfe83c06b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:53.712664 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7aa3ae2e-8f6b-44f7-b26a-8a5cfe83c06b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:53.766152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-81f95e82-851d-404f-b028-255676a54948 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:53.766946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-81f95e82-851d-404f-b028-255676a54948 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:53.775406 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-81f95e82-851d-404f-b028-255676a54948 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:53.775872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-81f95e82-851d-404f-b028-255676a54948 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:53.776662 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-81f95e82-851d-404f-b028-255676a54948 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:53.777130 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-81f95e82-851d-404f-b028-255676a54948 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:53.844981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-81f95e82-851d-404f-b028-255676a54948 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:53.855562 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-81f95e82-851d-404f-b028-255676a54948 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1511 microversion: 2.1 time: 0.093214 May 28 03:44:53.856555 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 553/1063] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:53 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1511 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:44:53.886322 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-ccb84c64-cad8-4c72-a51c-f35a7396887f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:53.886967 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-ccb84c64-cad8-4c72-a51c-f35a7396887f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:53.894976 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ccb84c64-cad8-4c72-a51c-f35a7396887f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:53.896761 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ccb84c64-cad8-4c72-a51c-f35a7396887f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:53.897932 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ccb84c64-cad8-4c72-a51c-f35a7396887f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:53.898576 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-ccb84c64-cad8-4c72-a51c-f35a7396887f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:53.977152 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-ccb84c64-cad8-4c72-a51c-f35a7396887f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:53.984841 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-ccb84c64-cad8-4c72-a51c-f35a7396887f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1200 microversion: 2.1 time: 0.104733 May 28 03:44:53.986308 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 554/1064] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:53 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1200 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:44:53.991160 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-5dea9c9c-6d87-4dea-92e7-8998fe0ec01a req-b0f0cf39-1857-47ce-9f8f-0ddfe1bb39d0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a799146f-de30-4095-8dfe-29d2b99837da", "name": "network-vif-deleted", "tag": "88c75168-a0ab-4da4-b23a-190a9097ec15"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:44:54.001141 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-5dea9c9c-6d87-4dea-92e7-8998fe0ec01a req-b0f0cf39-1857-47ce-9f8f-0ddfe1bb39d0 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:54.001666 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-5dea9c9c-6d87-4dea-92e7-8998fe0ec01a req-b0f0cf39-1857-47ce-9f8f-0ddfe1bb39d0 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:54.013417 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-5dea9c9c-6d87-4dea-92e7-8998fe0ec01a req-b0f0cf39-1857-47ce-9f8f-0ddfe1bb39d0 service nova] Creating event network-vif-deleted:88c75168-a0ab-4da4-b23a-190a9097ec15 for instance a799146f-de30-4095-8dfe-29d2b99837da on hv-cinder-84336 May 28 03:44:54.018880 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-5dea9c9c-6d87-4dea-92e7-8998fe0ec01a req-b0f0cf39-1857-47ce-9f8f-0ddfe1bb39d0 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031005 May 28 03:44:54.019418 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 555/1065] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:44:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:44:54.056262 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7aa3ae2e-8f6b-44f7-b26a-8a5cfe83c06b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:54.063806 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7aa3ae2e-8f6b-44f7-b26a-8a5cfe83c06b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1314 microversion: 2.1 time: 0.373069 May 28 03:44:54.064383 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 511/1066] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:53 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1314 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:54.875023 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-91e32277-9db7-4914-b3ab-a4b474b2077f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:54.876093 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-91e32277-9db7-4914-b3ab-a4b474b2077f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:54.894205 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-91e32277-9db7-4914-b3ab-a4b474b2077f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:54.894660 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-91e32277-9db7-4914-b3ab-a4b474b2077f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:54.895452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-91e32277-9db7-4914-b3ab-a4b474b2077f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:54.896400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-91e32277-9db7-4914-b3ab-a4b474b2077f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:54.955954 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-91e32277-9db7-4914-b3ab-a4b474b2077f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:54.959838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-91e32277-9db7-4914-b3ab-a4b474b2077f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:54.961529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-91e32277-9db7-4914-b3ab-a4b474b2077f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lazy-loading 'fault' on Instance uuid a799146f-de30-4095-8dfe-29d2b99837da {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:44:55.003526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-92d7ddf4-e1e9-453b-802f-381852c8fbd3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:55.004776 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-92d7ddf4-e1e9-453b-802f-381852c8fbd3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:55.016763 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-92d7ddf4-e1e9-453b-802f-381852c8fbd3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:55.027627 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-92d7ddf4-e1e9-453b-802f-381852c8fbd3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:55.027890 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-92d7ddf4-e1e9-453b-802f-381852c8fbd3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:55.028299 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-92d7ddf4-e1e9-453b-802f-381852c8fbd3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:55.073819 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-91e32277-9db7-4914-b3ab-a4b474b2077f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1a9600bad844f60937aa35126cd1be4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23f8194639b14942b36011698807b031', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:55.083665 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-91e32277-9db7-4914-b3ab-a4b474b2077f tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 200 len: 1348 microversion: 2.1 time: 0.212879 May 28 03:44:55.092781 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 556/1067] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:54 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 1348 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:55.092781 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-cb8030dd-2552-408f-9d0a-516eceb67ce5 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:55.092781 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-cb8030dd-2552-408f-9d0a-516eceb67ce5 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:55.105757 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb8030dd-2552-408f-9d0a-516eceb67ce5 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:55.106223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb8030dd-2552-408f-9d0a-516eceb67ce5 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:55.107018 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb8030dd-2552-408f-9d0a-516eceb67ce5 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:55.107463 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb8030dd-2552-408f-9d0a-516eceb67ce5 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:55.318252 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-92d7ddf4-e1e9-453b-802f-381852c8fbd3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:55.327606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-92d7ddf4-e1e9-453b-802f-381852c8fbd3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1278 microversion: 2.1 time: 0.328382 May 28 03:44:55.328177 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 512/1068] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:54 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1278 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:55.384546 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-cb8030dd-2552-408f-9d0a-516eceb67ce5 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:55.395077 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-cb8030dd-2552-408f-9d0a-516eceb67ce5 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1314 microversion: 2.1 time: 0.307475 May 28 03:44:55.395924 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 557/1069] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:55 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1314 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:55.534353 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-852277db-5e2c-422b-ac5b-b363672f33e3 req-be79c808-c49e-477e-a843-706f7617292f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "df122191-3ca2-4f3e-8076-99d5d5e713b0", "name": "network-vif-plugged", "status": "completed", "tag": "dd26d70b-fd3e-4adb-8cf1-11836f3ced7a"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:44:55.542611 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-852277db-5e2c-422b-ac5b-b363672f33e3 req-be79c808-c49e-477e-a843-706f7617292f service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:55.543051 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-852277db-5e2c-422b-ac5b-b363672f33e3 req-be79c808-c49e-477e-a843-706f7617292f service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:55.554464 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-852277db-5e2c-422b-ac5b-b363672f33e3 req-be79c808-c49e-477e-a843-706f7617292f service nova] Creating event network-vif-plugged:dd26d70b-fd3e-4adb-8cf1-11836f3ced7a for instance df122191-3ca2-4f3e-8076-99d5d5e713b0 on hv-cinder-84336 May 28 03:44:55.559582 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-852277db-5e2c-422b-ac5b-b363672f33e3 req-be79c808-c49e-477e-a843-706f7617292f service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028800 May 28 03:44:55.560416 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 513/1070] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:44:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:44:56.208086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-671d2c72-f8dd-4394-b754-f18bdfd7ec87 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:56.208086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-671d2c72-f8dd-4394-b754-f18bdfd7ec87 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] [instance: a799146f-de30-4095-8dfe-29d2b99837da] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:56.208086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-671d2c72-f8dd-4394-b754-f18bdfd7ec87 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:56.208086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-671d2c72-f8dd-4394-b754-f18bdfd7ec87 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-671d2c72-f8dd-4394-b754-f18bdfd7ec87 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-671d2c72-f8dd-4394-b754-f18bdfd7ec87 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-671d2c72-f8dd-4394-b754-f18bdfd7ec87 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] HTTP exception thrown: Instance a799146f-de30-4095-8dfe-29d2b99837da could not be found. May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-671d2c72-f8dd-4394-b754-f18bdfd7ec87 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Request method failure captured: May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da HTTP/1.1 May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:44:56.209587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZoxitRvHhrJeKeY-N2wgjumvGOZ5QlT8tA4Ny_zszUWSwzflpU1pG7cvrqv1qx56aTTz3CFK6Qn8toCs-3JjstETBtRUIUq7kE4jrGPlFnIzIJmoHSjzyDs7EcVNCw6QVT-xbdP0E5YM_AzwEZhTQXODVIHxOKA5OgVlER2W5zT9Eg May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: 23f8194639b14942b36011698807b031 May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-TestMinimumBasicScenario-681911436 May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/23f8194639b14942b36011698807b031"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/23f8194639b14942b36011698807b031"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/23f8194639b14942b36011698807b031"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_23f8194639b14942b36011698807b031"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-TestMinimumBasicScenario-681911436 May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: 23f8194639b14942b36011698807b031 May 28 03:44:56.212171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-TestMinimumBasicScenario-681911436 May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-TestMinimumBasicScenario-681911436-project May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: c1a9600bad844f60937aa35126cd1be4 May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-TestMinimumBasicScenario-681911436-project May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance a799146f-de30-4095-8dfe-29d2b99837da could not be found. May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': 'a799146f-de30-4095-8dfe-29d2b99837da'} May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a799146f-de30-4095-8dfe-29d2b99837da could not be found. May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:44:56.215355 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a799146f-de30-4095-8dfe-29d2b99837da could not be found. May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:44:56.217380 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:44:56.219031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:44:56.219031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a799146f-de30-4095-8dfe-29d2b99837da could not be found. May 28 03:44:56.219031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:44:56.219031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-671d2c72-f8dd-4394-b754-f18bdfd7ec87 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Returning 404 to user: Instance a799146f-de30-4095-8dfe-29d2b99837da could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:44:56.219031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-671d2c72-f8dd-4394-b754-f18bdfd7ec87 tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da" status: 404 len: 111 microversion: 2.1 time: 0.033941 May 28 03:44:56.219031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 558/1071] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:56 2022] GET /compute/v2.1/servers/a799146f-de30-4095-8dfe-29d2b99837da => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:44:56.219621 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b9dbdb00-3190-4458-b377-9f6c1452519e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] Calling method '.version_select at 0x7feb2c02f790>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:56.247393 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-b9dbdb00-3190-4458-b377-9f6c1452519e tempest-TestMinimumBasicScenario-681911436 tempest-TestMinimumBasicScenario-681911436-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1616322512" status: 202 len: 0 microversion: 2.1 time: 0.107476 May 28 03:44:56.249349 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 514/1072] 10.222.0.41 () {58 vars in 1271 bytes} [Sat May 28 03:44:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1616322512 => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:44:56.346223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-986a7fb6-9f09-4bd8-997e-d81bea7a262a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:56.347110 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-986a7fb6-9f09-4bd8-997e-d81bea7a262a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:56.354498 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-986a7fb6-9f09-4bd8-997e-d81bea7a262a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:56.355050 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-986a7fb6-9f09-4bd8-997e-d81bea7a262a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:56.355860 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-986a7fb6-9f09-4bd8-997e-d81bea7a262a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:56.356351 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-986a7fb6-9f09-4bd8-997e-d81bea7a262a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:56.416692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-2c788c86-b53a-469b-8d3a-2c0e015737fb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:56.417574 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-2c788c86-b53a-469b-8d3a-2c0e015737fb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:56.427762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2c788c86-b53a-469b-8d3a-2c0e015737fb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:56.440431 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2c788c86-b53a-469b-8d3a-2c0e015737fb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:56.444445 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2c788c86-b53a-469b-8d3a-2c0e015737fb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:56.445853 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-2c788c86-b53a-469b-8d3a-2c0e015737fb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:56.600177 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-986a7fb6-9f09-4bd8-997e-d81bea7a262a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:56.617841 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-986a7fb6-9f09-4bd8-997e-d81bea7a262a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.275442 May 28 03:44:56.626284 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 559/1073] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:56 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:44:56.762532 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-d74ba63e-e48a-4f76-b42a-5f7c75a4a654 req-a1325611-8c30-4610-8a64-580341c86cd0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8c6232c7-4a11-4f67-a028-96d05393e4df", "tag": "1049955a-bd14-461f-b144-ea03d2f89fe6"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:44:56.771272 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-d74ba63e-e48a-4f76-b42a-5f7c75a4a654 req-a1325611-8c30-4610-8a64-580341c86cd0 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:56.771716 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-d74ba63e-e48a-4f76-b42a-5f7c75a4a654 req-a1325611-8c30-4610-8a64-580341c86cd0 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:56.780800 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-2c788c86-b53a-469b-8d3a-2c0e015737fb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:56.792480 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-d74ba63e-e48a-4f76-b42a-5f7c75a4a654 req-a1325611-8c30-4610-8a64-580341c86cd0 service nova] Creating event network-changed:1049955a-bd14-461f-b144-ea03d2f89fe6 for instance 8c6232c7-4a11-4f67-a028-96d05393e4df on hv-cinder-84336 May 28 03:44:56.793288 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-2c788c86-b53a-469b-8d3a-2c0e015737fb tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1314 microversion: 2.1 time: 0.384698 May 28 03:44:56.803054 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 515/1074] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:56 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1314 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:56.813403 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-d74ba63e-e48a-4f76-b42a-5f7c75a4a654 req-a1325611-8c30-4610-8a64-580341c86cd0 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052886 May 28 03:44:56.813403 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 560/1075] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:44:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:44:57.650262 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0c7d90d1-08ee-4989-8cbf-bd44c65e2d0d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:57.651079 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0c7d90d1-08ee-4989-8cbf-bd44c65e2d0d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:57.659021 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c7d90d1-08ee-4989-8cbf-bd44c65e2d0d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:57.659466 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c7d90d1-08ee-4989-8cbf-bd44c65e2d0d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:57.660230 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c7d90d1-08ee-4989-8cbf-bd44c65e2d0d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:57.660658 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0c7d90d1-08ee-4989-8cbf-bd44c65e2d0d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:57.816452 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5705385b-1263-41e3-ac52-c75341618f42 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:57.818023 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5705385b-1263-41e3-ac52-c75341618f42 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:57.834362 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5705385b-1263-41e3-ac52-c75341618f42 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:57.834756 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5705385b-1263-41e3-ac52-c75341618f42 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:57.836880 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5705385b-1263-41e3-ac52-c75341618f42 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:57.837593 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5705385b-1263-41e3-ac52-c75341618f42 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:57.899089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0c7d90d1-08ee-4989-8cbf-bd44c65e2d0d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:57.908050 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0c7d90d1-08ee-4989-8cbf-bd44c65e2d0d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.261621 May 28 03:44:57.909333 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 516/1076] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:57 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:58.256715 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5705385b-1263-41e3-ac52-c75341618f42 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:58.267329 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5705385b-1263-41e3-ac52-c75341618f42 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1463 microversion: 2.1 time: 0.455275 May 28 03:44:58.268269 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 561/1077] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:57 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1463 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:58.307800 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-08a34686-0b99-4c5e-a633-bd830f0bd963 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:58.308607 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-08a34686-0b99-4c5e-a633-bd830f0bd963 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:58.315994 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-08a34686-0b99-4c5e-a633-bd830f0bd963 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:58.316648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-08a34686-0b99-4c5e-a633-bd830f0bd963 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:58.317472 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-08a34686-0b99-4c5e-a633-bd830f0bd963 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:58.317919 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-08a34686-0b99-4c5e-a633-bd830f0bd963 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:58.346021 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-08a34686-0b99-4c5e-a633-bd830f0bd963 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:44:58.363179 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:58.684002 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-08a34686-0b99-4c5e-a633-bd830f0bd963 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "DELETE /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 204 len: 0 microversion: 2.1 time: 0.379749 May 28 03:44:58.684833 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 517/1078] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:44:58 2022] DELETE /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 28 03:44:58.693612 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a00a3ee5-346c-41c6-bebb-aae56da3a2b8 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:58.694232 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a00a3ee5-346c-41c6-bebb-aae56da3a2b8 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:58.704574 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a00a3ee5-346c-41c6-bebb-aae56da3a2b8 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:58.705026 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a00a3ee5-346c-41c6-bebb-aae56da3a2b8 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:58.705840 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a00a3ee5-346c-41c6-bebb-aae56da3a2b8 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:58.706273 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a00a3ee5-346c-41c6-bebb-aae56da3a2b8 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:58.911750 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a00a3ee5-346c-41c6-bebb-aae56da3a2b8 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:58.919960 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a00a3ee5-346c-41c6-bebb-aae56da3a2b8 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1469 microversion: 2.1 time: 0.230578 May 28 03:44:58.920556 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 562/1079] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:58 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1469 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:44:58.926992 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-b35b3748-6c42-425a-9a2f-f7e17cb5e95d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:58.927692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-b35b3748-6c42-425a-9a2f-f7e17cb5e95d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:58.934124 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b35b3748-6c42-425a-9a2f-f7e17cb5e95d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:58.934564 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b35b3748-6c42-425a-9a2f-f7e17cb5e95d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:58.935314 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b35b3748-6c42-425a-9a2f-f7e17cb5e95d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:58.935836 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-b35b3748-6c42-425a-9a2f-f7e17cb5e95d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:58.961536 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:59.116848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-b35b3748-6c42-425a-9a2f-f7e17cb5e95d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:44:59.127043 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-b35b3748-6c42-425a-9a2f-f7e17cb5e95d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.204045 May 28 03:44:59.128588 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 518/1080] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:58 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:44:59.938089 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-335acc01-7d93-43df-aaac-159518834ea9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:44:59.943937 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-335acc01-7d93-43df-aaac-159518834ea9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:44:59.946769 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-335acc01-7d93-43df-aaac-159518834ea9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:59.947436 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-335acc01-7d93-43df-aaac-159518834ea9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:44:59.948218 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-335acc01-7d93-43df-aaac-159518834ea9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:44:59.948936 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-335acc01-7d93-43df-aaac-159518834ea9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:00.145858 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8e448915-d896-4c17-9e1d-89a330dc5008 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:00.146699 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8e448915-d896-4c17-9e1d-89a330dc5008 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:00.154340 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8e448915-d896-4c17-9e1d-89a330dc5008 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:00.157477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8e448915-d896-4c17-9e1d-89a330dc5008 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:00.157477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8e448915-d896-4c17-9e1d-89a330dc5008 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:00.157477 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8e448915-d896-4c17-9e1d-89a330dc5008 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:00.191932 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-335acc01-7d93-43df-aaac-159518834ea9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:00.200217 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-335acc01-7d93-43df-aaac-159518834ea9 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1469 microversion: 2.1 time: 0.265986 May 28 03:45:00.202835 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 563/1081] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:44:59 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1469 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:00.405948 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8e448915-d896-4c17-9e1d-89a330dc5008 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:00.426488 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8e448915-d896-4c17-9e1d-89a330dc5008 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.284479 May 28 03:45:00.429730 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 519/1082] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:00 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:01.025687 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-80940b44-7792-47c5-bbfb-ebd8fbc4e0ac req-fc34512a-0dda-46e1-a986-0c518390521b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "df122191-3ca2-4f3e-8076-99d5d5e713b0", "name": "network-vif-deleted", "tag": "dd26d70b-fd3e-4adb-8cf1-11836f3ced7a"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:45:01.037297 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-80940b44-7792-47c5-bbfb-ebd8fbc4e0ac req-fc34512a-0dda-46e1-a986-0c518390521b service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:01.037854 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-80940b44-7792-47c5-bbfb-ebd8fbc4e0ac req-fc34512a-0dda-46e1-a986-0c518390521b service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:01.052073 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-80940b44-7792-47c5-bbfb-ebd8fbc4e0ac req-fc34512a-0dda-46e1-a986-0c518390521b service nova] Creating event network-vif-deleted:dd26d70b-fd3e-4adb-8cf1-11836f3ced7a for instance df122191-3ca2-4f3e-8076-99d5d5e713b0 on hv-cinder-84336 May 28 03:45:01.058446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-80940b44-7792-47c5-bbfb-ebd8fbc4e0ac req-fc34512a-0dda-46e1-a986-0c518390521b service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039246 May 28 03:45:01.059841 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 564/1083] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:45:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:45:01.219050 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c3374dac-eec2-4995-a8f9-bad5c36a9d4b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:01.220137 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c3374dac-eec2-4995-a8f9-bad5c36a9d4b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:01.232935 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c3374dac-eec2-4995-a8f9-bad5c36a9d4b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:01.234608 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c3374dac-eec2-4995-a8f9-bad5c36a9d4b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:01.238958 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c3374dac-eec2-4995-a8f9-bad5c36a9d4b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:01.239526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c3374dac-eec2-4995-a8f9-bad5c36a9d4b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:01.344452 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=14 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:01.367389 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c3374dac-eec2-4995-a8f9-bad5c36a9d4b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20b48a7b54a84c569a1ba39148dc5e70', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '282a34892e974900926a37a76690258c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:01.386658 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c3374dac-eec2-4995-a8f9-bad5c36a9d4b tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 200 len: 1299 microversion: 2.1 time: 0.172544 May 28 03:45:01.387606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 520/1084] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:01 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 1299 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:01.451057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3bdb5f3c-2f70-4a5c-818b-8ae81075a9a7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:01.451760 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3bdb5f3c-2f70-4a5c-818b-8ae81075a9a7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:01.460339 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3bdb5f3c-2f70-4a5c-818b-8ae81075a9a7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:01.461424 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3bdb5f3c-2f70-4a5c-818b-8ae81075a9a7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:01.462565 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3bdb5f3c-2f70-4a5c-818b-8ae81075a9a7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:01.463129 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3bdb5f3c-2f70-4a5c-818b-8ae81075a9a7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:01.617904 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3bdb5f3c-2f70-4a5c-818b-8ae81075a9a7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:01.626466 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3bdb5f3c-2f70-4a5c-818b-8ae81075a9a7 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.181122 May 28 03:45:01.627058 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 565/1085] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:01 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:02.404557 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0b3e4097-21bf-4ef0-b12a-f8661bef5a40 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:02.405195 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0b3e4097-21bf-4ef0-b12a-f8661bef5a40 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] [instance: df122191-3ca2-4f3e-8076-99d5d5e713b0] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:02.416599 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0b3e4097-21bf-4ef0-b12a-f8661bef5a40 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:02.417070 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0b3e4097-21bf-4ef0-b12a-f8661bef5a40 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:02.417894 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0b3e4097-21bf-4ef0-b12a-f8661bef5a40 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:02.418525 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0b3e4097-21bf-4ef0-b12a-f8661bef5a40 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:02.435001 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-0b3e4097-21bf-4ef0-b12a-f8661bef5a40 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] HTTP exception thrown: Instance df122191-3ca2-4f3e-8076-99d5d5e713b0 could not be found. May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0b3e4097-21bf-4ef0-b12a-f8661bef5a40 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Request method failure captured: May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 HTTP/1.1 May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZqkX4mgG-rIU_Bk5poXbyOlmBoMDyapV9NH6yVtG-ZYhK-z25_PLF4f6FV0THXVDuDWiQ3DBFsqKWaDDYiqqb_nyr_Rwbx2C0zxJGs4t2raiPMCE50miLgUz68o_TAiZarNOiE9BGZIHgF3Xv4nIp8WTi1zAhMmg46W6YWSmsCVIuE May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: 282a34892e974900926a37a76690258c May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-VolumesNegativeTest-795744133 May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:45:02.436822 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/282a34892e974900926a37a76690258c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/282a34892e974900926a37a76690258c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/282a34892e974900926a37a76690258c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_282a34892e974900926a37a76690258c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-VolumesNegativeTest-795744133 May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: 282a34892e974900926a37a76690258c May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-VolumesNegativeTest-795744133 May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-VolumesNegativeTest-795744133-project May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 20b48a7b54a84c569a1ba39148dc5e70 May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-VolumesNegativeTest-795744133-project May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance df122191-3ca2-4f3e-8076-99d5d5e713b0 could not be found. May 28 03:45:02.438648 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': 'df122191-3ca2-4f3e-8076-99d5d5e713b0'} May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance df122191-3ca2-4f3e-8076-99d5d5e713b0 could not be found. May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:45:02.439518 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance df122191-3ca2-4f3e-8076-99d5d5e713b0 could not be found. May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance df122191-3ca2-4f3e-8076-99d5d5e713b0 could not be found. May 28 03:45:02.440771 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:45:02.441939 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0b3e4097-21bf-4ef0-b12a-f8661bef5a40 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] Returning 404 to user: Instance df122191-3ca2-4f3e-8076-99d5d5e713b0 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:45:02.441939 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0b3e4097-21bf-4ef0-b12a-f8661bef5a40 tempest-VolumesNegativeTest-795744133 tempest-VolumesNegativeTest-795744133-project] 10.222.0.41 "GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0" status: 404 len: 111 microversion: 2.1 time: 0.038024 May 28 03:45:02.441939 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 521/1086] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:02 2022] GET /compute/v2.1/servers/df122191-3ca2-4f3e-8076-99d5d5e713b0 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:45:02.644426 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-72a03f81-8597-4193-bfa7-5c261b8724a1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:02.645040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-72a03f81-8597-4193-bfa7-5c261b8724a1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:02.648728 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:02.652249 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-72a03f81-8597-4193-bfa7-5c261b8724a1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:02.652918 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-72a03f81-8597-4193-bfa7-5c261b8724a1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:02.653544 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-72a03f81-8597-4193-bfa7-5c261b8724a1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:02.654186 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-72a03f81-8597-4193-bfa7-5c261b8724a1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:02.681669 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=100 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:02.817005 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-72a03f81-8597-4193-bfa7-5c261b8724a1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:02.824336 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-72a03f81-8597-4193-bfa7-5c261b8724a1 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.183902 May 28 03:45:02.824909 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 566/1087] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:02 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:03.841617 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-710d1613-7698-4cc8-8309-9145162adf63 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:03.842235 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-710d1613-7698-4cc8-8309-9145162adf63 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:03.849962 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-710d1613-7698-4cc8-8309-9145162adf63 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:03.853868 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-710d1613-7698-4cc8-8309-9145162adf63 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:03.854838 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-710d1613-7698-4cc8-8309-9145162adf63 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:03.855294 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-710d1613-7698-4cc8-8309-9145162adf63 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:04.067485 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-710d1613-7698-4cc8-8309-9145162adf63 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:04.077796 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-710d1613-7698-4cc8-8309-9145162adf63 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.240001 May 28 03:45:04.079214 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 522/1088] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:03 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:05.096134 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-75f8db11-e07a-41d8-980f-07671875f615 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:05.097081 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-75f8db11-e07a-41d8-980f-07671875f615 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:05.105253 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-75f8db11-e07a-41d8-980f-07671875f615 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:05.105703 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-75f8db11-e07a-41d8-980f-07671875f615 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:05.106511 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-75f8db11-e07a-41d8-980f-07671875f615 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:05.106946 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-75f8db11-e07a-41d8-980f-07671875f615 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:05.291032 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-75f8db11-e07a-41d8-980f-07671875f615 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:05.301822 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-75f8db11-e07a-41d8-980f-07671875f615 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.209882 May 28 03:45:05.302953 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 567/1089] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:05 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:06.318449 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-a6119372-4300-4067-92c7-d28a00a14366 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:06.332398 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-a6119372-4300-4067-92c7-d28a00a14366 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:06.332398 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a6119372-4300-4067-92c7-d28a00a14366 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:06.332398 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a6119372-4300-4067-92c7-d28a00a14366 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:06.332398 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a6119372-4300-4067-92c7-d28a00a14366 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:06.332894 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-a6119372-4300-4067-92c7-d28a00a14366 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:06.484074 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-a6119372-4300-4067-92c7-d28a00a14366 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:06.496505 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-a6119372-4300-4067-92c7-d28a00a14366 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.181697 May 28 03:45:06.497091 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 523/1090] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:06 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:07.515045 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-03462cd8-aadf-43d1-a360-a422e723ed3f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:07.515737 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-03462cd8-aadf-43d1-a360-a422e723ed3f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:07.522384 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-03462cd8-aadf-43d1-a360-a422e723ed3f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:07.522838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-03462cd8-aadf-43d1-a360-a422e723ed3f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:07.523640 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-03462cd8-aadf-43d1-a360-a422e723ed3f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:07.524073 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-03462cd8-aadf-43d1-a360-a422e723ed3f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:07.666601 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-03462cd8-aadf-43d1-a360-a422e723ed3f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:07.674930 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-03462cd8-aadf-43d1-a360-a422e723ed3f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.163450 May 28 03:45:07.684876 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 568/1091] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:07 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:08.691702 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8571a8b8-c4a2-4fa5-ba78-7032b4771c0c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:08.692398 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8571a8b8-c4a2-4fa5-ba78-7032b4771c0c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:08.699101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8571a8b8-c4a2-4fa5-ba78-7032b4771c0c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:08.699544 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8571a8b8-c4a2-4fa5-ba78-7032b4771c0c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:08.700294 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8571a8b8-c4a2-4fa5-ba78-7032b4771c0c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:08.700722 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8571a8b8-c4a2-4fa5-ba78-7032b4771c0c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:08.838684 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8571a8b8-c4a2-4fa5-ba78-7032b4771c0c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:08.847368 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8571a8b8-c4a2-4fa5-ba78-7032b4771c0c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.159607 May 28 03:45:08.847885 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 524/1092] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:08 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:09.373373 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-f7ba8f01-20ad-499a-8370-737d5a8a4c87 req-69fc1e85-d5c2-4f7e-80a8-ab404f88f00d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8c6232c7-4a11-4f67-a028-96d05393e4df", "name": "network-vif-plugged", "status": "completed", "tag": "1049955a-bd14-461f-b144-ea03d2f89fe6"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:45:09.383398 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-f7ba8f01-20ad-499a-8370-737d5a8a4c87 req-69fc1e85-d5c2-4f7e-80a8-ab404f88f00d service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:09.383848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-f7ba8f01-20ad-499a-8370-737d5a8a4c87 req-69fc1e85-d5c2-4f7e-80a8-ab404f88f00d service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:09.407984 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-f7ba8f01-20ad-499a-8370-737d5a8a4c87 req-69fc1e85-d5c2-4f7e-80a8-ab404f88f00d service nova] Creating event network-vif-plugged:1049955a-bd14-461f-b144-ea03d2f89fe6 for instance 8c6232c7-4a11-4f67-a028-96d05393e4df on hv-cinder-84336 May 28 03:45:09.407984 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-f7ba8f01-20ad-499a-8370-737d5a8a4c87 req-69fc1e85-d5c2-4f7e-80a8-ab404f88f00d service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034629 May 28 03:45:09.407984 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 569/1093] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:45:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:45:09.864071 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e29c9d65-d774-49bf-8d73-ff8ae42e138a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:09.864936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e29c9d65-d774-49bf-8d73-ff8ae42e138a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:09.872599 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e29c9d65-d774-49bf-8d73-ff8ae42e138a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:09.876628 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e29c9d65-d774-49bf-8d73-ff8ae42e138a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:09.876628 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e29c9d65-d774-49bf-8d73-ff8ae42e138a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:09.876628 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e29c9d65-d774-49bf-8d73-ff8ae42e138a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:10.007164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e29c9d65-d774-49bf-8d73-ff8ae42e138a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:10.015229 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e29c9d65-d774-49bf-8d73-ff8ae42e138a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1266 microversion: 2.1 time: 0.155178 May 28 03:45:10.018317 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 525/1094] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:09 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:10.931810 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-2115377622", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1367140617", "security_groups": [{"name": "tempest-secgroup-smoke-1496361496"}], "networks": [{"uuid": "906b0a82-c757-4c32-a49b-21a69d65b543"}], "block_device_mapping_v2": [{"uuid": "a60d336a-a84a-449b-b40b-e8fd4eff7be8", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:45:11.033303 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e3a1db20-5b49-4233-b0dc-5416f7aeaae3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:11.034638 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e3a1db20-5b49-4233-b0dc-5416f7aeaae3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:11.042185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] validate_networks() for [('906b0a82-c757-4c32-a49b-21a69d65b543', None, None, None, None, None)] {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:45:11.048393 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=4,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:11.052153 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e3a1db20-5b49-4233-b0dc-5416f7aeaae3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:11.052677 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e3a1db20-5b49-4233-b0dc-5416f7aeaae3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:11.053591 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e3a1db20-5b49-4233-b0dc-5416f7aeaae3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:11.054650 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e3a1db20-5b49-4233-b0dc-5416f7aeaae3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:11.192326 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e3a1db20-5b49-4233-b0dc-5416f7aeaae3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:11.195849 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e3a1db20-5b49-4233-b0dc-5416f7aeaae3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1452 microversion: 2.1 time: 0.166622 May 28 03:45:11.196427 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 526/1095] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:11 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1452 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:11.211725 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1fd39abe-ad4e-40c1-9b57-5b996a510ed0 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:11.212324 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1fd39abe-ad4e-40c1-9b57-5b996a510ed0 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:11.218128 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1fd39abe-ad4e-40c1-9b57-5b996a510ed0 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:11.218570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1fd39abe-ad4e-40c1-9b57-5b996a510ed0 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:11.219395 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1fd39abe-ad4e-40c1-9b57-5b996a510ed0 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:11.219827 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1fd39abe-ad4e-40c1-9b57-5b996a510ed0 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:11.354312 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1fd39abe-ad4e-40c1-9b57-5b996a510ed0 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:11.361893 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1fd39abe-ad4e-40c1-9b57-5b996a510ed0 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1452 microversion: 2.1 time: 0.153944 May 28 03:45:11.362460 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 527/1096] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:11 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1452 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:45:11.414269 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:45:11.414694 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:45:11.415169 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:45:11.415577 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:45:11.415987 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:45:11.425042 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:11.425528 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:11.426356 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:11.426790 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:11.444128 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Getting quotas for project d1d9f51a3cc44062af827a498842ee80. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:45:11.449529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Getting quotas for user 303285e036734e5cbd491920ae7473d9 and project d1d9f51a3cc44062af827a498842ee80. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:45:11.505712 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:45:11.508367 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='a60d336a-a84a-449b-b40b-e8fd4eff7be8',volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:45:11.954775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-49ef1c65-ea88-45a8-aa56-a29ac5c87950 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.028064 May 28 03:45:11.960238 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 570/1097] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:45:10 2022] POST /compute/v2.1/servers => generated 406 bytes in 1034 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:45:11.966021 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-11f6e370-ccec-4d13-91fb-ecfb8ce79e6c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:11.966642 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-11f6e370-ccec-4d13-91fb-ecfb8ce79e6c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:12.035497 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-11f6e370-ccec-4d13-91fb-ecfb8ce79e6c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:12.043079 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-11f6e370-ccec-4d13-91fb-ecfb8ce79e6c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1089 microversion: 2.1 time: 0.080825 May 28 03:45:12.044053 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 528/1098] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:11 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1089 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:13.059799 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-247144e6-c99d-4558-9461-3e6209747e51 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:13.060411 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-247144e6-c99d-4558-9461-3e6209747e51 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:13.113853 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-247144e6-c99d-4558-9461-3e6209747e51 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:13.120934 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-247144e6-c99d-4558-9461-3e6209747e51 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1089 microversion: 2.1 time: 0.064991 May 28 03:45:13.121783 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 571/1099] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:13 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1089 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:13.470327 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-92e19e2e-d552-415b-bf24-10173130890b req-e1e0ace5-93e8-438c-babe-11e3004688b5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8c6232c7-4a11-4f67-a028-96d05393e4df", "tag": "1049955a-bd14-461f-b144-ea03d2f89fe6"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:45:13.480709 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-92e19e2e-d552-415b-bf24-10173130890b req-e1e0ace5-93e8-438c-babe-11e3004688b5 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:13.481236 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-92e19e2e-d552-415b-bf24-10173130890b req-e1e0ace5-93e8-438c-babe-11e3004688b5 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:13.492643 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-92e19e2e-d552-415b-bf24-10173130890b req-e1e0ace5-93e8-438c-babe-11e3004688b5 service nova] Creating event network-changed:1049955a-bd14-461f-b144-ea03d2f89fe6 for instance 8c6232c7-4a11-4f67-a028-96d05393e4df on hv-cinder-84336 May 28 03:45:13.497910 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-92e19e2e-d552-415b-bf24-10173130890b req-e1e0ace5-93e8-438c-babe-11e3004688b5 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031297 May 28 03:45:13.498536 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 529/1100] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:45:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:45:14.138582 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8083da4b-2110-4eba-8fda-e1e228e0f992 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:14.139211 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8083da4b-2110-4eba-8fda-e1e228e0f992 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:14.146985 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8083da4b-2110-4eba-8fda-e1e228e0f992 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:14.147711 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8083da4b-2110-4eba-8fda-e1e228e0f992 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:14.148711 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8083da4b-2110-4eba-8fda-e1e228e0f992 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:14.149377 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8083da4b-2110-4eba-8fda-e1e228e0f992 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:14.220112 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8083da4b-2110-4eba-8fda-e1e228e0f992 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:14.229305 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8083da4b-2110-4eba-8fda-e1e228e0f992 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1187 microversion: 2.1 time: 0.094145 May 28 03:45:14.230236 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 572/1101] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:14 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1187 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:15.248844 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7e1334ed-bd4f-4683-b7ea-8c19e0acdc5b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:15.249636 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7e1334ed-bd4f-4683-b7ea-8c19e0acdc5b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:15.256597 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7e1334ed-bd4f-4683-b7ea-8c19e0acdc5b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:15.257050 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7e1334ed-bd4f-4683-b7ea-8c19e0acdc5b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:15.258075 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7e1334ed-bd4f-4683-b7ea-8c19e0acdc5b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:15.258520 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7e1334ed-bd4f-4683-b7ea-8c19e0acdc5b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:15.512503 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7e1334ed-bd4f-4683-b7ea-8c19e0acdc5b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:15.520307 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7e1334ed-bd4f-4683-b7ea-8c19e0acdc5b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1273 microversion: 2.1 time: 0.276129 May 28 03:45:15.520955 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 530/1102] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:15 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1273 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:16.103419 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-0a1141d6-e46e-4c49-9b09-018c004d95ee req-5870b566-34a0-4794-bb78-9456f4787ff5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b37b6f57-630a-4342-bd89-6ccba324b98f", "tag": "791555af-21e2-4f77-8de6-77284ba93aae"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:45:16.111866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-0a1141d6-e46e-4c49-9b09-018c004d95ee req-5870b566-34a0-4794-bb78-9456f4787ff5 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:16.112313 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-0a1141d6-e46e-4c49-9b09-018c004d95ee req-5870b566-34a0-4794-bb78-9456f4787ff5 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:16.132642 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-0a1141d6-e46e-4c49-9b09-018c004d95ee req-5870b566-34a0-4794-bb78-9456f4787ff5 service nova] Creating event network-changed:791555af-21e2-4f77-8de6-77284ba93aae for instance b37b6f57-630a-4342-bd89-6ccba324b98f on hv-cinder-84336 May 28 03:45:16.138728 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-0a1141d6-e46e-4c49-9b09-018c004d95ee req-5870b566-34a0-4794-bb78-9456f4787ff5 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043889 May 28 03:45:16.139347 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 573/1103] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:45:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:45:16.538114 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-24ccb26f-5702-44d7-8be5-63ff41febd76 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:16.538738 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-24ccb26f-5702-44d7-8be5-63ff41febd76 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:16.546274 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-24ccb26f-5702-44d7-8be5-63ff41febd76 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:16.546730 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-24ccb26f-5702-44d7-8be5-63ff41febd76 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:16.547549 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-24ccb26f-5702-44d7-8be5-63ff41febd76 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:16.547985 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-24ccb26f-5702-44d7-8be5-63ff41febd76 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:16.929179 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-24ccb26f-5702-44d7-8be5-63ff41febd76 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:16.936658 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-24ccb26f-5702-44d7-8be5-63ff41febd76 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1261 microversion: 2.1 time: 0.402495 May 28 03:45:16.937259 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 531/1104] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:16 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1261 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:17.954247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-4213222f-7060-4bd1-802d-34608a112ed6 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:17.954957 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-4213222f-7060-4bd1-802d-34608a112ed6 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:17.961633 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4213222f-7060-4bd1-802d-34608a112ed6 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:17.962081 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4213222f-7060-4bd1-802d-34608a112ed6 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:17.962841 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4213222f-7060-4bd1-802d-34608a112ed6 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:17.963278 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-4213222f-7060-4bd1-802d-34608a112ed6 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:18.096675 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-4213222f-7060-4bd1-802d-34608a112ed6 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:18.104593 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-4213222f-7060-4bd1-802d-34608a112ed6 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1261 microversion: 2.1 time: 0.154506 May 28 03:45:18.105171 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 574/1105] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:17 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1261 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:19.122157 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e1b69078-202f-41b2-9e7c-bfbc2b0d06bc tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:19.122758 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e1b69078-202f-41b2-9e7c-bfbc2b0d06bc tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:19.129632 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e1b69078-202f-41b2-9e7c-bfbc2b0d06bc tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:19.130287 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e1b69078-202f-41b2-9e7c-bfbc2b0d06bc tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:19.131339 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e1b69078-202f-41b2-9e7c-bfbc2b0d06bc tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:19.131908 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e1b69078-202f-41b2-9e7c-bfbc2b0d06bc tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:19.264415 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e1b69078-202f-41b2-9e7c-bfbc2b0d06bc tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:19.271796 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e1b69078-202f-41b2-9e7c-bfbc2b0d06bc tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1261 microversion: 2.1 time: 0.154206 May 28 03:45:19.272357 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 532/1106] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:19 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1261 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:20.287623 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1e74251c-cf87-47a0-8cef-9bd41eec3e89 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:20.288234 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1e74251c-cf87-47a0-8cef-9bd41eec3e89 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:20.294660 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1e74251c-cf87-47a0-8cef-9bd41eec3e89 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:20.295101 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1e74251c-cf87-47a0-8cef-9bd41eec3e89 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:20.295850 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1e74251c-cf87-47a0-8cef-9bd41eec3e89 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:20.296280 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1e74251c-cf87-47a0-8cef-9bd41eec3e89 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:20.423419 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1e74251c-cf87-47a0-8cef-9bd41eec3e89 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:20.430938 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1e74251c-cf87-47a0-8cef-9bd41eec3e89 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1261 microversion: 2.1 time: 0.147083 May 28 03:45:20.431504 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 575/1107] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:20 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1261 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:21.446761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-77f53362-516a-4b89-8a37-9d199a5722a1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:21.447396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-77f53362-516a-4b89-8a37-9d199a5722a1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:21.453922 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-77f53362-516a-4b89-8a37-9d199a5722a1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:21.454485 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-77f53362-516a-4b89-8a37-9d199a5722a1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:21.455245 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-77f53362-516a-4b89-8a37-9d199a5722a1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:21.455670 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-77f53362-516a-4b89-8a37-9d199a5722a1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:21.582769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-77f53362-516a-4b89-8a37-9d199a5722a1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:21.591598 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-77f53362-516a-4b89-8a37-9d199a5722a1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1261 microversion: 2.1 time: 0.148701 May 28 03:45:21.592215 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 533/1108] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:21 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1261 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:22.607720 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-803b1b3c-800c-466f-bda7-f4e11abf657f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:22.608247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-803b1b3c-800c-466f-bda7-f4e11abf657f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:22.610664 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=2 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:22.615492 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-803b1b3c-800c-466f-bda7-f4e11abf657f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:22.615955 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-803b1b3c-800c-466f-bda7-f4e11abf657f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:22.616708 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-803b1b3c-800c-466f-bda7-f4e11abf657f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:22.617168 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-803b1b3c-800c-466f-bda7-f4e11abf657f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:22.754040 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-803b1b3c-800c-466f-bda7-f4e11abf657f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:22.761995 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-803b1b3c-800c-466f-bda7-f4e11abf657f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1261 microversion: 2.1 time: 0.158180 May 28 03:45:22.765182 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 576/1109] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:22 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1261 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:23.778009 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e2938dc8-8693-43da-8967-8ac7af5a79b1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:23.778542 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e2938dc8-8693-43da-8967-8ac7af5a79b1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:23.780481 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=2 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:23.785861 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e2938dc8-8693-43da-8967-8ac7af5a79b1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:23.786311 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e2938dc8-8693-43da-8967-8ac7af5a79b1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:23.787074 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e2938dc8-8693-43da-8967-8ac7af5a79b1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:23.787515 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e2938dc8-8693-43da-8967-8ac7af5a79b1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:23.914697 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e2938dc8-8693-43da-8967-8ac7af5a79b1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:23.922216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e2938dc8-8693-43da-8967-8ac7af5a79b1 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1261 microversion: 2.1 time: 0.148116 May 28 03:45:23.922779 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 534/1110] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:23 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1261 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:24.938476 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d9c19c59-783e-4595-a01f-d205a8f7f5bd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:24.939299 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d9c19c59-783e-4595-a01f-d205a8f7f5bd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:24.941425 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=8 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:24.948090 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9c19c59-783e-4595-a01f-d205a8f7f5bd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:24.948532 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9c19c59-783e-4595-a01f-d205a8f7f5bd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:24.949320 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9c19c59-783e-4595-a01f-d205a8f7f5bd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:24.949833 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d9c19c59-783e-4595-a01f-d205a8f7f5bd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:25.080183 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d9c19c59-783e-4595-a01f-d205a8f7f5bd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:25.087654 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d9c19c59-783e-4595-a01f-d205a8f7f5bd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1261 microversion: 2.1 time: 0.153007 May 28 03:45:25.088215 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 577/1111] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:24 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1261 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:26.104521 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-feda099f-6096-4ee8-b020-adfe9425d037 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:26.105064 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-feda099f-6096-4ee8-b020-adfe9425d037 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:26.111522 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-feda099f-6096-4ee8-b020-adfe9425d037 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:26.111967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-feda099f-6096-4ee8-b020-adfe9425d037 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:26.112724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-feda099f-6096-4ee8-b020-adfe9425d037 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:26.113183 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-feda099f-6096-4ee8-b020-adfe9425d037 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:26.247789 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-feda099f-6096-4ee8-b020-adfe9425d037 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:26.255236 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-feda099f-6096-4ee8-b020-adfe9425d037 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1261 microversion: 2.1 time: 0.155384 May 28 03:45:26.255792 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 535/1112] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:26 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1261 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:27.271808 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-eb8e364f-3d3e-46aa-b574-fffe0eaa4d32 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:27.272413 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-eb8e364f-3d3e-46aa-b574-fffe0eaa4d32 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:27.279436 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-eb8e364f-3d3e-46aa-b574-fffe0eaa4d32 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:27.279919 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-eb8e364f-3d3e-46aa-b574-fffe0eaa4d32 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:27.280672 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-eb8e364f-3d3e-46aa-b574-fffe0eaa4d32 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:27.281126 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-eb8e364f-3d3e-46aa-b574-fffe0eaa4d32 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:27.421947 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-eb8e364f-3d3e-46aa-b574-fffe0eaa4d32 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:27.429513 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-eb8e364f-3d3e-46aa-b574-fffe0eaa4d32 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1261 microversion: 2.1 time: 0.161649 May 28 03:45:27.430607 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 578/1113] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:27 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1261 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:27.462991 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-d62d0dce-c883-4623-879d-d0bf32324039 req-2b3cd64e-a5f8-4e72-9dcc-74e7902e313b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b37b6f57-630a-4342-bd89-6ccba324b98f", "name": "network-vif-plugged", "status": "completed", "tag": "791555af-21e2-4f77-8de6-77284ba93aae"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:45:27.470068 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-d62d0dce-c883-4623-879d-d0bf32324039 req-2b3cd64e-a5f8-4e72-9dcc-74e7902e313b service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:27.470512 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-d62d0dce-c883-4623-879d-d0bf32324039 req-2b3cd64e-a5f8-4e72-9dcc-74e7902e313b service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:27.480988 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-d62d0dce-c883-4623-879d-d0bf32324039 req-2b3cd64e-a5f8-4e72-9dcc-74e7902e313b service nova] Creating event network-vif-plugged:791555af-21e2-4f77-8de6-77284ba93aae for instance b37b6f57-630a-4342-bd89-6ccba324b98f on hv-cinder-84336 May 28 03:45:27.486401 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-d62d0dce-c883-4623-879d-d0bf32324039 req-2b3cd64e-a5f8-4e72-9dcc-74e7902e313b service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028524 May 28 03:45:27.491480 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 536/1114] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:45:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:45:28.447561 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-17b8047f-151a-42f4-9c0f-0b7b17d79db8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:28.448175 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-17b8047f-151a-42f4-9c0f-0b7b17d79db8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:28.454716 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17b8047f-151a-42f4-9c0f-0b7b17d79db8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:28.455169 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17b8047f-151a-42f4-9c0f-0b7b17d79db8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:28.455931 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17b8047f-151a-42f4-9c0f-0b7b17d79db8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:28.456369 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17b8047f-151a-42f4-9c0f-0b7b17d79db8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:28.585550 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-17b8047f-151a-42f4-9c0f-0b7b17d79db8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:28.593807 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-17b8047f-151a-42f4-9c0f-0b7b17d79db8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1261 microversion: 2.1 time: 0.150803 May 28 03:45:28.594383 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 579/1115] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:28 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1261 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:29.611139 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0f32e88a-0e5b-48a0-a2a8-8273eff0b617 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:29.611833 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0f32e88a-0e5b-48a0-a2a8-8273eff0b617 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:29.619207 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0f32e88a-0e5b-48a0-a2a8-8273eff0b617 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:29.620248 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0f32e88a-0e5b-48a0-a2a8-8273eff0b617 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:29.620433 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0f32e88a-0e5b-48a0-a2a8-8273eff0b617 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:29.620877 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0f32e88a-0e5b-48a0-a2a8-8273eff0b617 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:29.762826 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0f32e88a-0e5b-48a0-a2a8-8273eff0b617 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:29.774527 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0f32e88a-0e5b-48a0-a2a8-8273eff0b617 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1445 microversion: 2.1 time: 0.167575 May 28 03:45:29.775101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 537/1116] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:29 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1445 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:29.790529 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-89d87d77-4f11-4414-8bda-8157729b101a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:45:29.791203 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-89d87d77-4f11-4414-8bda-8157729b101a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:45:29.797661 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-89d87d77-4f11-4414-8bda-8157729b101a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:29.798100 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-89d87d77-4f11-4414-8bda-8157729b101a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:29.798848 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-89d87d77-4f11-4414-8bda-8157729b101a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:29.799277 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-89d87d77-4f11-4414-8bda-8157729b101a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:29.927644 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-89d87d77-4f11-4414-8bda-8157729b101a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:45:29.935713 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-89d87d77-4f11-4414-8bda-8157729b101a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1445 microversion: 2.1 time: 0.148965 May 28 03:45:29.936274 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 580/1117] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:45:29 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1445 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:45:32.017890 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-807e42ac-d37f-4219-91f6-74d240ae5d5b req-6e603ebd-9093-4aa1-a6e1-bc693ac960fa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b37b6f57-630a-4342-bd89-6ccba324b98f", "tag": "791555af-21e2-4f77-8de6-77284ba93aae"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:45:32.031686 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-807e42ac-d37f-4219-91f6-74d240ae5d5b req-6e603ebd-9093-4aa1-a6e1-bc693ac960fa service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:45:32.032134 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-807e42ac-d37f-4219-91f6-74d240ae5d5b req-6e603ebd-9093-4aa1-a6e1-bc693ac960fa service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:45:32.035111 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=99 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:32.045389 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-807e42ac-d37f-4219-91f6-74d240ae5d5b req-6e603ebd-9093-4aa1-a6e1-bc693ac960fa service nova] Creating event network-changed:791555af-21e2-4f77-8de6-77284ba93aae for instance b37b6f57-630a-4342-bd89-6ccba324b98f on hv-cinder-84336 May 28 03:45:32.050731 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-807e42ac-d37f-4219-91f6-74d240ae5d5b req-6e603ebd-9093-4aa1-a6e1-bc693ac960fa service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036678 May 28 03:45:32.051404 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 538/1118] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:45:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:45:52.442781 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:45:52.443320 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Sat May 28 03:45:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:47:00.930841 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR oslo.messaging._drivers.impl_rabbit [-] [b8e30501-ec68-4ddf-a703-3de320a70c5c] AMQP server on 10.222.0.41:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: May 28 03:47:00.931543 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=43,nova_api:INSERT=3 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:00.932288 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=68 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:00.936269 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1881455196", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1597361276", "security_groups": [{"name": "tempest-secgroup-smoke-1971281779"}], "block_device_mapping_v2": [{"uuid": "6f01af71-b409-49c3-8ae5-86ec2c47bc3b", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "af6636fe-9088-4d20-9a1e-fa3df2392948"}]}} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:47:01.168964 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.network.neutron [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] validate_networks() for [('af6636fe-9088-4d20-9a1e-fa3df2392948', None, None, None, None, None)] {{(pid=124604) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 28 03:47:01.536263 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Flavor limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 28 03:47:01.536682 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Image limits 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 28 03:47:01.537197 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Flavor pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 28 03:47:01.537592 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Image pref 0:0:0 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 28 03:47:01.537997 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.virt.hardware [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124604) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 28 03:47:01.547141 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:01.547605 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:01.548389 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:01.548825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:01.566120 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Getting quotas for project c0fe831f6e954e05b7d448f1d4f43754. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 28 03:47:01.569954 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.quota [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Getting quotas for user 2b515cebf18643c38f1cfc2e3fad1db4 and project c0fe831f6e954e05b7d448f1d4f43754. Resources: {'cores', 'instances', 'ram'} {{(pid=124604) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 28 03:47:01.626324 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Going to run 1 instances... {{(pid=124604) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 28 03:47:01.628446 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='6f01af71-b409-49c3-8ae5-86ec2c47bc3b',volume_size=None,volume_type=None)] {{(pid=124604) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 28 03:47:01.939057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO oslo.messaging._drivers.impl_rabbit [-] [b8e30501-ec68-4ddf-a703-3de320a70c5c] Reconnected to AMQP server on 10.222.0.41:5672 via [amqp] client with port 37042. May 28 03:47:01.992289 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-9f1bdb0c-1896-43eb-bc84-11c4da5d51d4 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.063394 May 28 03:47:01.992925 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 581/1119] 10.222.0.41 () {60 vars in 1146 bytes} [Sat May 28 03:47:00 2022] POST /compute/v2.1/servers => generated 406 bytes in 1065 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 28 03:47:02.002198 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=27 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:02.007744 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d1250bb0-ba07-4b2c-bf2d-5127ebc1859e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:02.014457 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d1250bb0-ba07-4b2c-bf2d-5127ebc1859e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:02.122519 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d1250bb0-ba07-4b2c-bf2d-5127ebc1859e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:02.129630 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d1250bb0-ba07-4b2c-bf2d-5127ebc1859e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1095 microversion: 2.1 time: 0.130132 May 28 03:47:02.130218 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 539/1120] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:01 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1095 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:47:03.148751 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1980478f-a8f8-48f7-9b42-7319e4df13d3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:03.149387 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1980478f-a8f8-48f7-9b42-7319e4df13d3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:03.203851 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1980478f-a8f8-48f7-9b42-7319e4df13d3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:03.210519 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1980478f-a8f8-48f7-9b42-7319e4df13d3 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1095 microversion: 2.1 time: 0.066724 May 28 03:47:03.211077 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 582/1121] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:03 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1095 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:04.227813 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-ee31706a-b796-4d65-ac55-3d46725c054c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:04.228529 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-ee31706a-b796-4d65-ac55-3d46725c054c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:04.287361 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-ee31706a-b796-4d65-ac55-3d46725c054c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:04.298102 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-ee31706a-b796-4d65-ac55-3d46725c054c tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1095 microversion: 2.1 time: 0.074117 May 28 03:47:04.300129 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 540/1122] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:04 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1095 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:05.315706 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bb299c0e-df78-4b76-abe7-995a86e23b1e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:05.316317 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bb299c0e-df78-4b76-abe7-995a86e23b1e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:05.371485 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bb299c0e-df78-4b76-abe7-995a86e23b1e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:05.378872 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bb299c0e-df78-4b76-abe7-995a86e23b1e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1095 microversion: 2.1 time: 0.067497 May 28 03:47:05.379440 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 583/1123] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:05 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1095 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:06.396274 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-49115efb-c45f-4ccb-95be-2eab1219b05a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:06.396878 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-49115efb-c45f-4ccb-95be-2eab1219b05a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:06.403885 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-49115efb-c45f-4ccb-95be-2eab1219b05a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:06.404328 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-49115efb-c45f-4ccb-95be-2eab1219b05a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:06.405083 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-49115efb-c45f-4ccb-95be-2eab1219b05a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:06.405535 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-49115efb-c45f-4ccb-95be-2eab1219b05a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:06.464636 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-49115efb-c45f-4ccb-95be-2eab1219b05a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:06.472506 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-49115efb-c45f-4ccb-95be-2eab1219b05a tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1145 microversion: 2.1 time: 0.080181 May 28 03:47:06.473063 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 541/1124] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:06 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1145 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:07.490057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6a4c3da6-7f79-4e5c-921b-3b232b2da1fe tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:07.490634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-6a4c3da6-7f79-4e5c-921b-3b232b2da1fe tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:07.497775 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6a4c3da6-7f79-4e5c-921b-3b232b2da1fe tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:07.498485 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6a4c3da6-7f79-4e5c-921b-3b232b2da1fe tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:07.499235 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6a4c3da6-7f79-4e5c-921b-3b232b2da1fe tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:07.499665 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6a4c3da6-7f79-4e5c-921b-3b232b2da1fe tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:07.561121 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6a4c3da6-7f79-4e5c-921b-3b232b2da1fe tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:07.568867 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6a4c3da6-7f79-4e5c-921b-3b232b2da1fe tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1193 microversion: 2.1 time: 0.083591 May 28 03:47:07.569480 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 584/1125] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:07 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1193 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:08.586032 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cf215b1c-b82d-4592-ac35-66c945291dc5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:08.586797 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-cf215b1c-b82d-4592-ac35-66c945291dc5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:08.593876 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cf215b1c-b82d-4592-ac35-66c945291dc5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:08.594434 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cf215b1c-b82d-4592-ac35-66c945291dc5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:08.595359 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cf215b1c-b82d-4592-ac35-66c945291dc5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:08.595865 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cf215b1c-b82d-4592-ac35-66c945291dc5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:08.790830 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-cf215b1c-b82d-4592-ac35-66c945291dc5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:08.799243 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-cf215b1c-b82d-4592-ac35-66c945291dc5 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1279 microversion: 2.1 time: 0.218194 May 28 03:47:08.799818 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 542/1126] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:08 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1279 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:09.623483 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-82ff560f-bb4e-4dec-a88d-df15d0e89087 req-a624444d-b668-4585-a5c0-226321a70e30 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ba87599-510f-4138-92f7-e909bb7df417", "tag": "11df9db6-964f-4442-9b85-d9da3d305d3c"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:47:09.633020 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-82ff560f-bb4e-4dec-a88d-df15d0e89087 req-a624444d-b668-4585-a5c0-226321a70e30 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:09.633784 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-82ff560f-bb4e-4dec-a88d-df15d0e89087 req-a624444d-b668-4585-a5c0-226321a70e30 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:09.646400 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-82ff560f-bb4e-4dec-a88d-df15d0e89087 req-a624444d-b668-4585-a5c0-226321a70e30 service nova] Creating event network-changed:11df9db6-964f-4442-9b85-d9da3d305d3c for instance 6ba87599-510f-4138-92f7-e909bb7df417 on hv-cinder-84336 May 28 03:47:09.651960 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-82ff560f-bb4e-4dec-a88d-df15d0e89087 req-a624444d-b668-4585-a5c0-226321a70e30 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.101094 May 28 03:47:09.652838 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 585/1127] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:47:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:47:09.815711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-87a4a3e0-7c47-4079-ae07-6e2c2393e70e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:09.816320 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-87a4a3e0-7c47-4079-ae07-6e2c2393e70e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:09.822809 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87a4a3e0-7c47-4079-ae07-6e2c2393e70e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:09.823251 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87a4a3e0-7c47-4079-ae07-6e2c2393e70e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:09.824006 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87a4a3e0-7c47-4079-ae07-6e2c2393e70e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:09.824434 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-87a4a3e0-7c47-4079-ae07-6e2c2393e70e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:10.053756 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-87a4a3e0-7c47-4079-ae07-6e2c2393e70e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:10.061132 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-87a4a3e0-7c47-4079-ae07-6e2c2393e70e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1267 microversion: 2.1 time: 0.249226 May 28 03:47:10.061873 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 543/1128] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:09 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1267 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:11.080470 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-44968e60-09e4-46a0-b23c-90f6a8c9bbf2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:11.081369 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-44968e60-09e4-46a0-b23c-90f6a8c9bbf2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:11.087676 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-44968e60-09e4-46a0-b23c-90f6a8c9bbf2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:11.088126 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-44968e60-09e4-46a0-b23c-90f6a8c9bbf2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:11.088897 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-44968e60-09e4-46a0-b23c-90f6a8c9bbf2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:11.089357 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-44968e60-09e4-46a0-b23c-90f6a8c9bbf2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:11.224556 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-44968e60-09e4-46a0-b23c-90f6a8c9bbf2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:11.231993 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-44968e60-09e4-46a0-b23c-90f6a8c9bbf2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1267 microversion: 2.1 time: 0.156962 May 28 03:47:11.232553 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 586/1129] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:11 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1267 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:12.249041 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-89853d12-9e1b-45a2-ae69-2740a2338760 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:12.249712 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-89853d12-9e1b-45a2-ae69-2740a2338760 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:12.256372 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-89853d12-9e1b-45a2-ae69-2740a2338760 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:12.256819 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-89853d12-9e1b-45a2-ae69-2740a2338760 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:12.257596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-89853d12-9e1b-45a2-ae69-2740a2338760 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:12.258027 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-89853d12-9e1b-45a2-ae69-2740a2338760 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:12.387802 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-89853d12-9e1b-45a2-ae69-2740a2338760 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:12.395199 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-89853d12-9e1b-45a2-ae69-2740a2338760 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1267 microversion: 2.1 time: 0.150378 May 28 03:47:12.395769 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 544/1130] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:12 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1267 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:13.411668 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-cb89b16c-250e-46f5-b5a9-bbaa80277dd9 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:13.412296 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-cb89b16c-250e-46f5-b5a9-bbaa80277dd9 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:13.414489 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=2 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:13.419864 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb89b16c-250e-46f5-b5a9-bbaa80277dd9 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:13.423564 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb89b16c-250e-46f5-b5a9-bbaa80277dd9 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:13.423564 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb89b16c-250e-46f5-b5a9-bbaa80277dd9 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:13.423564 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-cb89b16c-250e-46f5-b5a9-bbaa80277dd9 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:13.553881 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-cb89b16c-250e-46f5-b5a9-bbaa80277dd9 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:13.561189 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-cb89b16c-250e-46f5-b5a9-bbaa80277dd9 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1267 microversion: 2.1 time: 0.153364 May 28 03:47:13.561742 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 587/1131] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:13 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:14.578319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5a430917-6714-4500-8eb0-1aed510fc67f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:14.578928 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5a430917-6714-4500-8eb0-1aed510fc67f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:14.581201 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell0:SELECT=4 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:14.586042 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5a430917-6714-4500-8eb0-1aed510fc67f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:14.590101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5a430917-6714-4500-8eb0-1aed510fc67f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:14.590101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5a430917-6714-4500-8eb0-1aed510fc67f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:14.590101 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5a430917-6714-4500-8eb0-1aed510fc67f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:14.722715 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5a430917-6714-4500-8eb0-1aed510fc67f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:14.730124 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5a430917-6714-4500-8eb0-1aed510fc67f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1267 microversion: 2.1 time: 0.155631 May 28 03:47:14.730674 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 545/1132] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:14 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:15.747618 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1f81126e-42be-435d-b5c2-4c30d22c18f2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:15.748489 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1f81126e-42be-435d-b5c2-4c30d22c18f2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:15.750885 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell0:SELECT=10 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:15.756457 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f81126e-42be-435d-b5c2-4c30d22c18f2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:15.756906 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f81126e-42be-435d-b5c2-4c30d22c18f2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:15.757701 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f81126e-42be-435d-b5c2-4c30d22c18f2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:15.758141 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1f81126e-42be-435d-b5c2-4c30d22c18f2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:15.886407 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1f81126e-42be-435d-b5c2-4c30d22c18f2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:15.893830 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1f81126e-42be-435d-b5c2-4c30d22c18f2 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1267 microversion: 2.1 time: 0.150045 May 28 03:47:15.894915 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 588/1133] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:15 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1267 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:16.910694 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-8b1d5e77-5a1d-47d6-bd0b-e91e2b3897b8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:16.911304 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-8b1d5e77-5a1d-47d6-bd0b-e91e2b3897b8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:16.917772 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8b1d5e77-5a1d-47d6-bd0b-e91e2b3897b8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:16.918213 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8b1d5e77-5a1d-47d6-bd0b-e91e2b3897b8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:16.918967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8b1d5e77-5a1d-47d6-bd0b-e91e2b3897b8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:16.919396 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-8b1d5e77-5a1d-47d6-bd0b-e91e2b3897b8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:17.047513 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-8b1d5e77-5a1d-47d6-bd0b-e91e2b3897b8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:17.054940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-8b1d5e77-5a1d-47d6-bd0b-e91e2b3897b8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1267 microversion: 2.1 time: 0.148254 May 28 03:47:17.055500 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 546/1134] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:16 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1267 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:18.072080 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d7d1785a-626d-43bf-846b-a3c2b165c632 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:18.072687 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d7d1785a-626d-43bf-846b-a3c2b165c632 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:18.079225 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d7d1785a-626d-43bf-846b-a3c2b165c632 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:18.079671 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d7d1785a-626d-43bf-846b-a3c2b165c632 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:18.080430 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d7d1785a-626d-43bf-846b-a3c2b165c632 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:18.080860 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d7d1785a-626d-43bf-846b-a3c2b165c632 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:18.220603 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d7d1785a-626d-43bf-846b-a3c2b165c632 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:18.228694 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d7d1785a-626d-43bf-846b-a3c2b165c632 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1267 microversion: 2.1 time: 0.160983 May 28 03:47:18.229293 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 589/1135] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:18 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1267 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:19.245936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0a3c9a53-53fa-4e85-aa40-0cc8dc30f2f8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:19.246465 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0a3c9a53-53fa-4e85-aa40-0cc8dc30f2f8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:19.253240 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0a3c9a53-53fa-4e85-aa40-0cc8dc30f2f8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:19.253724 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0a3c9a53-53fa-4e85-aa40-0cc8dc30f2f8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:19.254483 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0a3c9a53-53fa-4e85-aa40-0cc8dc30f2f8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:19.254916 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0a3c9a53-53fa-4e85-aa40-0cc8dc30f2f8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:19.384921 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-0a3c9a53-53fa-4e85-aa40-0cc8dc30f2f8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:19.392421 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0a3c9a53-53fa-4e85-aa40-0cc8dc30f2f8 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1267 microversion: 2.1 time: 0.150421 May 28 03:47:19.392970 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 547/1136] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:19 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1267 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:19.901917 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-318d33d1-a9b8-49f5-8c87-af201281e497 req-1eacc374-4551-4b87-ac1c-435c5d3590d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6ba87599-510f-4138-92f7-e909bb7df417", "name": "network-vif-plugged", "status": "completed", "tag": "11df9db6-964f-4442-9b85-d9da3d305d3c"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:47:19.910498 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-318d33d1-a9b8-49f5-8c87-af201281e497 req-1eacc374-4551-4b87-ac1c-435c5d3590d8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:19.915118 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-318d33d1-a9b8-49f5-8c87-af201281e497 req-1eacc374-4551-4b87-ac1c-435c5d3590d8 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:19.922265 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-318d33d1-a9b8-49f5-8c87-af201281e497 req-1eacc374-4551-4b87-ac1c-435c5d3590d8 service nova] Creating event network-vif-plugged:11df9db6-964f-4442-9b85-d9da3d305d3c for instance 6ba87599-510f-4138-92f7-e909bb7df417 on hv-cinder-84336 May 28 03:47:19.928079 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-318d33d1-a9b8-49f5-8c87-af201281e497 req-1eacc374-4551-4b87-ac1c-435c5d3590d8 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083953 May 28 03:47:19.928672 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 590/1137] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:47:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:47:20.409063 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-85382a35-c393-42ee-afb2-ada332b58d9f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:20.409671 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-85382a35-c393-42ee-afb2-ada332b58d9f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:20.416760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-85382a35-c393-42ee-afb2-ada332b58d9f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:20.417242 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-85382a35-c393-42ee-afb2-ada332b58d9f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:20.418009 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-85382a35-c393-42ee-afb2-ada332b58d9f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:20.418443 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-85382a35-c393-42ee-afb2-ada332b58d9f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:20.546471 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-85382a35-c393-42ee-afb2-ada332b58d9f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:20.554089 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-85382a35-c393-42ee-afb2-ada332b58d9f tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1267 microversion: 2.1 time: 0.148961 May 28 03:47:20.554657 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 548/1138] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:20 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1267 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:21.571737 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-d5b5b527-d49d-4b42-9990-cdb37ece440d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:21.572477 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-d5b5b527-d49d-4b42-9990-cdb37ece440d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:21.579383 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5b5b527-d49d-4b42-9990-cdb37ece440d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:21.579770 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5b5b527-d49d-4b42-9990-cdb37ece440d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:21.580533 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5b5b527-d49d-4b42-9990-cdb37ece440d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:21.580970 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-d5b5b527-d49d-4b42-9990-cdb37ece440d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:21.716312 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-d5b5b527-d49d-4b42-9990-cdb37ece440d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:21.724514 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-d5b5b527-d49d-4b42-9990-cdb37ece440d tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1453 microversion: 2.1 time: 0.156909 May 28 03:47:21.725092 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 591/1139] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:21 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1453 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:21.741292 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-1ee3418e-12c2-480f-a363-784522440d7e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:21.741899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-1ee3418e-12c2-480f-a363-784522440d7e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:21.748525 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ee3418e-12c2-480f-a363-784522440d7e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:21.748985 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ee3418e-12c2-480f-a363-784522440d7e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:21.749786 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ee3418e-12c2-480f-a363-784522440d7e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:21.750219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-1ee3418e-12c2-480f-a363-784522440d7e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:21.882552 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-1ee3418e-12c2-480f-a363-784522440d7e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:21.890093 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-1ee3418e-12c2-480f-a363-784522440d7e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1453 microversion: 2.1 time: 0.152619 May 28 03:47:21.890660 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 549/1140] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:21 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1453 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:23.999353 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-27a70242-811f-4ede-94f9-59f98071d038 req-fb5f2349-e5f4-4bf5-86e6-6f91a50bf194 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ba87599-510f-4138-92f7-e909bb7df417", "tag": "11df9db6-964f-4442-9b85-d9da3d305d3c"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:47:24.007998 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-27a70242-811f-4ede-94f9-59f98071d038 req-fb5f2349-e5f4-4bf5-86e6-6f91a50bf194 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:24.008434 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-27a70242-811f-4ede-94f9-59f98071d038 req-fb5f2349-e5f4-4bf5-86e6-6f91a50bf194 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:24.019958 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-27a70242-811f-4ede-94f9-59f98071d038 req-fb5f2349-e5f4-4bf5-86e6-6f91a50bf194 service nova] Creating event network-changed:11df9db6-964f-4442-9b85-d9da3d305d3c for instance 6ba87599-510f-4138-92f7-e909bb7df417 on hv-cinder-84336 May 28 03:47:24.025460 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-27a70242-811f-4ede-94f9-59f98071d038 req-fb5f2349-e5f4-4bf5-86e6-6f91a50bf194 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029899 May 28 03:47:24.026070 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 592/1141] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:47:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:47:27.301896 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-544e3a96-c7f0-4b04-a0ba-6d560ecda532 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:47:27.302500 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-544e3a96-c7f0-4b04-a0ba-6d560ecda532 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:27.311733 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-544e3a96-c7f0-4b04-a0ba-6d560ecda532 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:27.312188 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-544e3a96-c7f0-4b04-a0ba-6d560ecda532 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:27.312964 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-544e3a96-c7f0-4b04-a0ba-6d560ecda532 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:27.313426 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-544e3a96-c7f0-4b04-a0ba-6d560ecda532 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:27.339192 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.objects.instance [None req-544e3a96-c7f0-4b04-a0ba-6d560ecda532 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lazy-loading 'flavor' on Instance uuid b37b6f57-630a-4342-bd89-6ccba324b98f {{(pid=124605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:47:27.526256 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-544e3a96-c7f0-4b04-a0ba-6d560ecda532 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "POST /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f/action" status: 202 len: 0 microversion: 2.1 time: 0.228193 May 28 03:47:27.526887 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 550/1142] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:47:27 2022] POST /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f/action => generated 0 bytes in 230 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:47:27.536570 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bf04e5da-fb4e-40d6-8b74-1d85b6194a2d tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:27.537324 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bf04e5da-fb4e-40d6-8b74-1d85b6194a2d tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:27.543556 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bf04e5da-fb4e-40d6-8b74-1d85b6194a2d tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:27.544043 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bf04e5da-fb4e-40d6-8b74-1d85b6194a2d tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:27.544762 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bf04e5da-fb4e-40d6-8b74-1d85b6194a2d tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:27.545246 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bf04e5da-fb4e-40d6-8b74-1d85b6194a2d tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:27.676930 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bf04e5da-fb4e-40d6-8b74-1d85b6194a2d tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:27.684993 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bf04e5da-fb4e-40d6-8b74-1d85b6194a2d tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.153078 May 28 03:47:27.685582 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 593/1143] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:27 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:28.703554 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-920a6d93-79af-43ef-8f8c-5fcce7998972 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:28.704170 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-920a6d93-79af-43ef-8f8c-5fcce7998972 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:28.711314 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-920a6d93-79af-43ef-8f8c-5fcce7998972 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:28.711752 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-920a6d93-79af-43ef-8f8c-5fcce7998972 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:28.712507 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-920a6d93-79af-43ef-8f8c-5fcce7998972 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:28.712932 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-920a6d93-79af-43ef-8f8c-5fcce7998972 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:28.843262 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-920a6d93-79af-43ef-8f8c-5fcce7998972 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:28.850669 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-920a6d93-79af-43ef-8f8c-5fcce7998972 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.151498 May 28 03:47:28.851232 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 551/1144] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:28 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:29.868203 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-df1eb255-e5ca-46e2-956c-8e67dac5baef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:29.868804 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-df1eb255-e5ca-46e2-956c-8e67dac5baef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:29.875573 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-df1eb255-e5ca-46e2-956c-8e67dac5baef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:29.876008 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-df1eb255-e5ca-46e2-956c-8e67dac5baef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:29.876758 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-df1eb255-e5ca-46e2-956c-8e67dac5baef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:29.877208 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-df1eb255-e5ca-46e2-956c-8e67dac5baef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:30.007341 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-df1eb255-e5ca-46e2-956c-8e67dac5baef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:30.014910 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-df1eb255-e5ca-46e2-956c-8e67dac5baef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.151055 May 28 03:47:30.015479 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 594/1145] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:29 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:31.032579 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7df97661-80a7-41bd-9967-7fde86a0564c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:31.033210 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7df97661-80a7-41bd-9967-7fde86a0564c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:31.040246 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7df97661-80a7-41bd-9967-7fde86a0564c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:31.040685 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7df97661-80a7-41bd-9967-7fde86a0564c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:31.041467 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7df97661-80a7-41bd-9967-7fde86a0564c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:31.041898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7df97661-80a7-41bd-9967-7fde86a0564c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:31.174526 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-7df97661-80a7-41bd-9967-7fde86a0564c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:31.181897 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7df97661-80a7-41bd-9967-7fde86a0564c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.153246 May 28 03:47:31.182446 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 552/1146] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:31 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:31.488990 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:47:31.489219 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Sat May 28 03:47:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:47:32.199118 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-8d5fef3f-5279-4f59-bce8-f890d0a39868 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:32.199723 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-8d5fef3f-5279-4f59-bce8-f890d0a39868 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:32.201695 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=46,nova_api:INSERT=3 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:32.206425 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d5fef3f-5279-4f59-bce8-f890d0a39868 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:32.206868 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d5fef3f-5279-4f59-bce8-f890d0a39868 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:32.207629 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d5fef3f-5279-4f59-bce8-f890d0a39868 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:32.208064 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-8d5fef3f-5279-4f59-bce8-f890d0a39868 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:32.212444 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=57 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:32.336699 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-8d5fef3f-5279-4f59-bce8-f890d0a39868 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:32.344178 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-8d5fef3f-5279-4f59-bce8-f890d0a39868 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.149271 May 28 03:47:32.344745 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 595/1147] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:32 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:33.362641 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9a6ba160-27db-4140-b835-9ec4d42633b8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:33.363233 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9a6ba160-27db-4140-b835-9ec4d42633b8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:33.365556 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=33 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:33.371417 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a6ba160-27db-4140-b835-9ec4d42633b8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:33.371860 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a6ba160-27db-4140-b835-9ec4d42633b8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:33.372631 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a6ba160-27db-4140-b835-9ec4d42633b8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:33.373075 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9a6ba160-27db-4140-b835-9ec4d42633b8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:33.377367 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=83,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:33.502654 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-9a6ba160-27db-4140-b835-9ec4d42633b8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:33.510878 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9a6ba160-27db-4140-b835-9ec4d42633b8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.152134 May 28 03:47:33.511460 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 553/1148] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:33 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:34.528375 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b0ffc352-88cb-4d8f-93fa-522800813d63 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:34.529015 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b0ffc352-88cb-4d8f-93fa-522800813d63 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:34.535524 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b0ffc352-88cb-4d8f-93fa-522800813d63 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:34.535965 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b0ffc352-88cb-4d8f-93fa-522800813d63 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:34.536713 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b0ffc352-88cb-4d8f-93fa-522800813d63 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:34.537170 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b0ffc352-88cb-4d8f-93fa-522800813d63 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:34.667616 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b0ffc352-88cb-4d8f-93fa-522800813d63 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:34.674949 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b0ffc352-88cb-4d8f-93fa-522800813d63 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.150487 May 28 03:47:34.675506 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 596/1149] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:34 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:35.691733 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3f1e5513-6714-404f-adef-0e31f8b6ec09 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:35.692332 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3f1e5513-6714-404f-adef-0e31f8b6ec09 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:35.699319 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3f1e5513-6714-404f-adef-0e31f8b6ec09 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:35.699756 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3f1e5513-6714-404f-adef-0e31f8b6ec09 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:35.700504 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3f1e5513-6714-404f-adef-0e31f8b6ec09 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:35.700931 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3f1e5513-6714-404f-adef-0e31f8b6ec09 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:35.832641 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3f1e5513-6714-404f-adef-0e31f8b6ec09 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:35.840766 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3f1e5513-6714-404f-adef-0e31f8b6ec09 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.152864 May 28 03:47:35.841354 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 554/1150] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:35 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:36.858127 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b90e02f3-ef1d-475b-8b1b-1cc16c0dbc8b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:36.858739 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b90e02f3-ef1d-475b-8b1b-1cc16c0dbc8b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:36.865162 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b90e02f3-ef1d-475b-8b1b-1cc16c0dbc8b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:36.865609 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b90e02f3-ef1d-475b-8b1b-1cc16c0dbc8b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:36.866371 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b90e02f3-ef1d-475b-8b1b-1cc16c0dbc8b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:36.866805 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b90e02f3-ef1d-475b-8b1b-1cc16c0dbc8b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:37.010363 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b90e02f3-ef1d-475b-8b1b-1cc16c0dbc8b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:37.017880 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b90e02f3-ef1d-475b-8b1b-1cc16c0dbc8b tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.163599 May 28 03:47:37.018445 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 597/1151] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:36 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:38.035407 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-3de6610f-5a3b-426b-acd6-2c01512e3260 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:38.036049 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-3de6610f-5a3b-426b-acd6-2c01512e3260 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:38.042792 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3de6610f-5a3b-426b-acd6-2c01512e3260 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:38.043238 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3de6610f-5a3b-426b-acd6-2c01512e3260 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:38.044000 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3de6610f-5a3b-426b-acd6-2c01512e3260 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:38.044432 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-3de6610f-5a3b-426b-acd6-2c01512e3260 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:38.271272 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-3de6610f-5a3b-426b-acd6-2c01512e3260 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:38.279236 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-3de6610f-5a3b-426b-acd6-2c01512e3260 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.247638 May 28 03:47:38.279805 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 555/1152] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:38 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:39.296247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-24f957c9-c97b-4aee-89d3-e23f3062b4ca tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:39.296859 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-24f957c9-c97b-4aee-89d3-e23f3062b4ca tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:39.303366 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24f957c9-c97b-4aee-89d3-e23f3062b4ca tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:39.303811 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24f957c9-c97b-4aee-89d3-e23f3062b4ca tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:39.305286 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24f957c9-c97b-4aee-89d3-e23f3062b4ca tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:39.305725 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-24f957c9-c97b-4aee-89d3-e23f3062b4ca tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:39.433551 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-24f957c9-c97b-4aee-89d3-e23f3062b4ca tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:39.440949 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-24f957c9-c97b-4aee-89d3-e23f3062b4ca tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.148543 May 28 03:47:39.441542 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 598/1153] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:39 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:40.457881 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-26684961-c48a-4838-a231-6eec3e3a3a5e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:40.458487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-26684961-c48a-4838-a231-6eec3e3a3a5e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:40.464812 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26684961-c48a-4838-a231-6eec3e3a3a5e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:40.469892 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26684961-c48a-4838-a231-6eec3e3a3a5e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:40.469892 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26684961-c48a-4838-a231-6eec3e3a3a5e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:40.469892 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-26684961-c48a-4838-a231-6eec3e3a3a5e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:40.598203 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-26684961-c48a-4838-a231-6eec3e3a3a5e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:40.605737 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-26684961-c48a-4838-a231-6eec3e3a3a5e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.151565 May 28 03:47:40.606310 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 556/1154] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:40 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:41.623285 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a9d4b9eb-ff2f-4c11-b970-c3c2bc68be45 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:41.624150 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a9d4b9eb-ff2f-4c11-b970-c3c2bc68be45 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:41.630634 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a9d4b9eb-ff2f-4c11-b970-c3c2bc68be45 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:41.631082 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a9d4b9eb-ff2f-4c11-b970-c3c2bc68be45 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:41.631840 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a9d4b9eb-ff2f-4c11-b970-c3c2bc68be45 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:41.632275 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a9d4b9eb-ff2f-4c11-b970-c3c2bc68be45 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:42.004985 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a9d4b9eb-ff2f-4c11-b970-c3c2bc68be45 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:42.012542 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a9d4b9eb-ff2f-4c11-b970-c3c2bc68be45 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1511 microversion: 2.1 time: 0.393073 May 28 03:47:42.013122 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 599/1155] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:41 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1511 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:42.029390 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-9bfdd831-4d3d-41c3-a6b2-9ee049092b1a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:47:42.029967 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-9bfdd831-4d3d-41c3-a6b2-9ee049092b1a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:42.036452 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9bfdd831-4d3d-41c3-a6b2-9ee049092b1a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:42.036968 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9bfdd831-4d3d-41c3-a6b2-9ee049092b1a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:42.037762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9bfdd831-4d3d-41c3-a6b2-9ee049092b1a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:42.038204 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-9bfdd831-4d3d-41c3-a6b2-9ee049092b1a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:42.295826 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-9bfdd831-4d3d-41c3-a6b2-9ee049092b1a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "POST /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f/action" status: 202 len: 0 microversion: 2.1 time: 0.270276 May 28 03:47:42.296432 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 557/1156] 10.222.0.41 () {60 vars in 1277 bytes} [Sat May 28 03:47:42 2022] POST /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f/action => generated 0 bytes in 272 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:47:42.304333 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-17bac405-f7be-4eb2-a2f9-02b6a06adc4f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:42.305684 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-17bac405-f7be-4eb2-a2f9-02b6a06adc4f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:42.312099 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17bac405-f7be-4eb2-a2f9-02b6a06adc4f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:42.312547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17bac405-f7be-4eb2-a2f9-02b6a06adc4f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:42.313344 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17bac405-f7be-4eb2-a2f9-02b6a06adc4f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:42.313779 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-17bac405-f7be-4eb2-a2f9-02b6a06adc4f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:42.451989 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-17bac405-f7be-4eb2-a2f9-02b6a06adc4f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:42.459482 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-17bac405-f7be-4eb2-a2f9-02b6a06adc4f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1519 microversion: 2.1 time: 0.158905 May 28 03:47:42.460047 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 600/1157] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:42 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1519 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:47:42.778370 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:47:42.779298 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Sat May 28 03:47:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:47:43.476690 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-898dfac8-93dc-4be5-ae12-f5d60b0c14ef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:43.477390 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-898dfac8-93dc-4be5-ae12-f5d60b0c14ef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:43.483320 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-898dfac8-93dc-4be5-ae12-f5d60b0c14ef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:43.483759 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-898dfac8-93dc-4be5-ae12-f5d60b0c14ef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:43.484513 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-898dfac8-93dc-4be5-ae12-f5d60b0c14ef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:43.484946 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-898dfac8-93dc-4be5-ae12-f5d60b0c14ef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:43.613264 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-898dfac8-93dc-4be5-ae12-f5d60b0c14ef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:43.621365 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-898dfac8-93dc-4be5-ae12-f5d60b0c14ef tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1521 microversion: 2.1 time: 0.148871 May 28 03:47:43.621920 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 558/1158] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:43 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1521 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:43.909311 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-322d7b8c-a737-42d9-ab0b-c101913efe8a req-89162648-ebca-4599-8f43-3245a029d8bd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b37b6f57-630a-4342-bd89-6ccba324b98f", "name": "network-vif-unplugged", "status": "completed", "tag": "791555af-21e2-4f77-8de6-77284ba93aae"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:47:43.921233 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-322d7b8c-a737-42d9-ab0b-c101913efe8a req-89162648-ebca-4599-8f43-3245a029d8bd service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:43.921233 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-322d7b8c-a737-42d9-ab0b-c101913efe8a req-89162648-ebca-4599-8f43-3245a029d8bd service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:43.932825 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.compute.server_external_events [req-322d7b8c-a737-42d9-ab0b-c101913efe8a req-89162648-ebca-4599-8f43-3245a029d8bd service nova] Unable to find a host for instance b37b6f57-630a-4342-bd89-6ccba324b98f. Dropping event network-vif-unplugged {{(pid=124604) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} May 28 03:47:43.938442 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-322d7b8c-a737-42d9-ab0b-c101913efe8a req-89162648-ebca-4599-8f43-3245a029d8bd service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.028609 May 28 03:47:43.938442 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 601/1159] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:47:43 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 30 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) May 28 03:47:44.641300 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-09f7f268-116d-4cf0-9550-c7a867c83a8f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:44.641899 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-09f7f268-116d-4cf0-9550-c7a867c83a8f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:44.653297 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-09f7f268-116d-4cf0-9550-c7a867c83a8f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:44.654061 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-09f7f268-116d-4cf0-9550-c7a867c83a8f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:44.655356 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-09f7f268-116d-4cf0-9550-c7a867c83a8f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:44.656087 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-09f7f268-116d-4cf0-9550-c7a867c83a8f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:44.801935 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-09f7f268-116d-4cf0-9550-c7a867c83a8f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:44.811596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-09f7f268-116d-4cf0-9550-c7a867c83a8f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1577 microversion: 2.1 time: 0.174100 May 28 03:47:44.812167 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 559/1160] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:44 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1577 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:45.833801 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1ee94274-056a-4800-944e-b4c28b20870e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:45.834396 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1ee94274-056a-4800-944e-b4c28b20870e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:45.840895 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1ee94274-056a-4800-944e-b4c28b20870e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:45.841362 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1ee94274-056a-4800-944e-b4c28b20870e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:45.842118 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1ee94274-056a-4800-944e-b4c28b20870e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:45.842547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1ee94274-056a-4800-944e-b4c28b20870e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:45.973315 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1ee94274-056a-4800-944e-b4c28b20870e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:45.980886 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1ee94274-056a-4800-944e-b4c28b20870e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1577 microversion: 2.1 time: 0.150925 May 28 03:47:45.981621 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 602/1161] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:45 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1577 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:46.998916 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-43fcb566-4721-4506-8104-de9f8adbda48 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:46.999487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-43fcb566-4721-4506-8104-de9f8adbda48 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:47.010457 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-43fcb566-4721-4506-8104-de9f8adbda48 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:47.011287 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-43fcb566-4721-4506-8104-de9f8adbda48 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:47.012533 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-43fcb566-4721-4506-8104-de9f8adbda48 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:47.013088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-43fcb566-4721-4506-8104-de9f8adbda48 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:47.140139 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-43fcb566-4721-4506-8104-de9f8adbda48 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:47.147574 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-43fcb566-4721-4506-8104-de9f8adbda48 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1577 microversion: 2.1 time: 0.153300 May 28 03:47:47.148131 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 560/1162] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:46 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1577 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:47:48.165338 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b1b93439-96a5-4203-992f-4957dfdf391c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:48.165948 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b1b93439-96a5-4203-992f-4957dfdf391c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:48.172289 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b1b93439-96a5-4203-992f-4957dfdf391c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:48.172746 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b1b93439-96a5-4203-992f-4957dfdf391c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:48.173538 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b1b93439-96a5-4203-992f-4957dfdf391c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:48.173983 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b1b93439-96a5-4203-992f-4957dfdf391c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:48.299847 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b1b93439-96a5-4203-992f-4957dfdf391c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:48.307302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b1b93439-96a5-4203-992f-4957dfdf391c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1577 microversion: 2.1 time: 0.146514 May 28 03:47:48.307876 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 603/1163] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:48 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1577 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:49.323765 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-e7c71c3d-d024-4d4a-8fee-a584ea05ff70 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:49.324377 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-e7c71c3d-d024-4d4a-8fee-a584ea05ff70 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:49.331001 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7c71c3d-d024-4d4a-8fee-a584ea05ff70 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:49.331451 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7c71c3d-d024-4d4a-8fee-a584ea05ff70 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:49.332231 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7c71c3d-d024-4d4a-8fee-a584ea05ff70 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:49.332668 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-e7c71c3d-d024-4d4a-8fee-a584ea05ff70 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:49.465455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-e7c71c3d-d024-4d4a-8fee-a584ea05ff70 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:49.472871 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-e7c71c3d-d024-4d4a-8fee-a584ea05ff70 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1577 microversion: 2.1 time: 0.153068 May 28 03:47:49.473458 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 561/1164] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:49 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1577 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:50.490489 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-09f8ea5d-e0ff-4b61-85ff-d05e4a848c54 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:50.491032 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-09f8ea5d-e0ff-4b61-85ff-d05e4a848c54 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:50.498252 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09f8ea5d-e0ff-4b61-85ff-d05e4a848c54 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:50.498950 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09f8ea5d-e0ff-4b61-85ff-d05e4a848c54 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:50.499717 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09f8ea5d-e0ff-4b61-85ff-d05e4a848c54 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:50.500154 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-09f8ea5d-e0ff-4b61-85ff-d05e4a848c54 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:50.634720 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-09f8ea5d-e0ff-4b61-85ff-d05e4a848c54 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:50.642274 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-09f8ea5d-e0ff-4b61-85ff-d05e4a848c54 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1577 microversion: 2.1 time: 0.155730 May 28 03:47:50.642832 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 604/1165] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:50 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1577 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:51.658884 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-c398b41c-4109-4cef-ba0a-491812a25b3f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:51.659451 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-c398b41c-4109-4cef-ba0a-491812a25b3f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:51.666034 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c398b41c-4109-4cef-ba0a-491812a25b3f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:51.666506 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c398b41c-4109-4cef-ba0a-491812a25b3f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:51.667273 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c398b41c-4109-4cef-ba0a-491812a25b3f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:51.667711 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-c398b41c-4109-4cef-ba0a-491812a25b3f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:51.796806 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-c398b41c-4109-4cef-ba0a-491812a25b3f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:51.804289 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-c398b41c-4109-4cef-ba0a-491812a25b3f tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1577 microversion: 2.1 time: 0.149551 May 28 03:47:51.807922 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 562/1166] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:51 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1577 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:52.822275 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a34a3c2f-5a8c-4866-9150-f2edcfbe27d8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:52.822893 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-a34a3c2f-5a8c-4866-9150-f2edcfbe27d8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:52.829416 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a34a3c2f-5a8c-4866-9150-f2edcfbe27d8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:52.829866 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a34a3c2f-5a8c-4866-9150-f2edcfbe27d8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:52.830630 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a34a3c2f-5a8c-4866-9150-f2edcfbe27d8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:52.831067 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-a34a3c2f-5a8c-4866-9150-f2edcfbe27d8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:52.968240 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-a34a3c2f-5a8c-4866-9150-f2edcfbe27d8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:52.975782 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a34a3c2f-5a8c-4866-9150-f2edcfbe27d8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1577 microversion: 2.1 time: 0.157355 May 28 03:47:52.976533 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 605/1167] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:52 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1577 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:53.993550 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-14abbff4-46a1-4da9-8467-68dcda8259e5 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:53.994841 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-14abbff4-46a1-4da9-8467-68dcda8259e5 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:54.001606 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-14abbff4-46a1-4da9-8467-68dcda8259e5 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:54.002057 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-14abbff4-46a1-4da9-8467-68dcda8259e5 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:54.002829 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-14abbff4-46a1-4da9-8467-68dcda8259e5 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:54.003268 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-14abbff4-46a1-4da9-8467-68dcda8259e5 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:54.137592 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-14abbff4-46a1-4da9-8467-68dcda8259e5 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:54.145158 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-14abbff4-46a1-4da9-8467-68dcda8259e5 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1577 microversion: 2.1 time: 0.155619 May 28 03:47:54.146465 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 563/1168] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:53 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1577 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:55.163797 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-5adfd06c-b1cb-4297-bcca-c465ba12125a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:55.164339 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-5adfd06c-b1cb-4297-bcca-c465ba12125a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:55.171295 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5adfd06c-b1cb-4297-bcca-c465ba12125a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:55.171748 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5adfd06c-b1cb-4297-bcca-c465ba12125a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:55.172503 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5adfd06c-b1cb-4297-bcca-c465ba12125a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:55.172938 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-5adfd06c-b1cb-4297-bcca-c465ba12125a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:55.305724 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-5adfd06c-b1cb-4297-bcca-c465ba12125a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:55.313425 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-5adfd06c-b1cb-4297-bcca-c465ba12125a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1577 microversion: 2.1 time: 0.153523 May 28 03:47:55.313987 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 606/1169] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:55 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1577 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:56.016609 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-e96cbf38-8642-4338-a059-49cbc1b5f8b6 req-06545b3c-a151-4ee0-b2be-db5b526575f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b37b6f57-630a-4342-bd89-6ccba324b98f", "name": "network-vif-plugged", "status": "completed", "tag": "791555af-21e2-4f77-8de6-77284ba93aae"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:47:56.025773 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-e96cbf38-8642-4338-a059-49cbc1b5f8b6 req-06545b3c-a151-4ee0-b2be-db5b526575f7 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:56.026379 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-e96cbf38-8642-4338-a059-49cbc1b5f8b6 req-06545b3c-a151-4ee0-b2be-db5b526575f7 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:56.040450 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-e96cbf38-8642-4338-a059-49cbc1b5f8b6 req-06545b3c-a151-4ee0-b2be-db5b526575f7 service nova] Creating event network-vif-plugged:791555af-21e2-4f77-8de6-77284ba93aae for instance b37b6f57-630a-4342-bd89-6ccba324b98f on hv-cinder-84336 May 28 03:47:56.047026 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-e96cbf38-8642-4338-a059-49cbc1b5f8b6 req-06545b3c-a151-4ee0-b2be-db5b526575f7 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034471 May 28 03:47:56.047641 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 564/1170] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:47:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:47:56.331361 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-97e54f6c-433a-4a3e-a1d3-0eb54589c0dd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:56.332019 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-97e54f6c-433a-4a3e-a1d3-0eb54589c0dd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:56.339055 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-97e54f6c-433a-4a3e-a1d3-0eb54589c0dd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:56.339706 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-97e54f6c-433a-4a3e-a1d3-0eb54589c0dd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:56.340275 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-97e54f6c-433a-4a3e-a1d3-0eb54589c0dd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:56.340895 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-97e54f6c-433a-4a3e-a1d3-0eb54589c0dd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:56.473037 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-97e54f6c-433a-4a3e-a1d3-0eb54589c0dd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:56.480622 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-97e54f6c-433a-4a3e-a1d3-0eb54589c0dd tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1577 microversion: 2.1 time: 0.153151 May 28 03:47:56.481206 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 607/1171] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:56 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1577 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:47:57.499334 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-cab5fd88-9a23-4b30-a2be-2d499803f5aa tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:47:57.500025 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-cab5fd88-9a23-4b30-a2be-2d499803f5aa tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:47:57.506246 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cab5fd88-9a23-4b30-a2be-2d499803f5aa tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:57.509570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cab5fd88-9a23-4b30-a2be-2d499803f5aa tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:57.509570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cab5fd88-9a23-4b30-a2be-2d499803f5aa tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:47:57.509570 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-cab5fd88-9a23-4b30-a2be-2d499803f5aa tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:47:57.639991 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-cab5fd88-9a23-4b30-a2be-2d499803f5aa tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:47:57.647468 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-cab5fd88-9a23-4b30-a2be-2d499803f5aa tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1564 microversion: 2.1 time: 0.153692 May 28 03:47:57.648038 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 565/1172] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:47:57 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1564 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:48:52.116700 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-88eedcdf-3bf2-406c-a25e-1480bfa5452b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:48:52.117282 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-88eedcdf-3bf2-406c-a25e-1480bfa5452b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:48:52.119302 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_api:SELECT=25 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:48:52.119524 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-307ddff1-4581-468f-85e1-a87a0c5ce741 req-ebda45f5-7abd-4fc0-8a7d-b2bc41de5527 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ba87599-510f-4138-92f7-e909bb7df417", "tag": "11df9db6-964f-4442-9b85-d9da3d305d3c"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:48:52.120053 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG dbcounter [-] [124604] Writing DB stats nova_cell1:SELECT=73 {{(pid=124604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:48:52.123827 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_api:SELECT=27 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:48:52.125056 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG dbcounter [-] [124605] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:48:52.125644 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-88eedcdf-3bf2-406c-a25e-1480bfa5452b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:52.126094 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-88eedcdf-3bf2-406c-a25e-1480bfa5452b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:52.126865 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-88eedcdf-3bf2-406c-a25e-1480bfa5452b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:52.127536 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-88eedcdf-3bf2-406c-a25e-1480bfa5452b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:52.130692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-307ddff1-4581-468f-85e1-a87a0c5ce741 req-ebda45f5-7abd-4fc0-8a7d-b2bc41de5527 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:52.131148 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-307ddff1-4581-468f-85e1-a87a0c5ce741 req-ebda45f5-7abd-4fc0-8a7d-b2bc41de5527 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:52.143970 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-307ddff1-4581-468f-85e1-a87a0c5ce741 req-ebda45f5-7abd-4fc0-8a7d-b2bc41de5527 service nova] Creating event network-changed:11df9db6-964f-4442-9b85-d9da3d305d3c for instance 6ba87599-510f-4138-92f7-e909bb7df417 on hv-cinder-84336 May 28 03:48:52.149738 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-307ddff1-4581-468f-85e1-a87a0c5ce741 req-ebda45f5-7abd-4fc0-8a7d-b2bc41de5527 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034681 May 28 03:48:52.150487 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 566/1173] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:48:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:48:52.157546 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-88eedcdf-3bf2-406c-a25e-1480bfa5452b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Going to try to terminate instance {{(pid=124604) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:48:52.447580 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-88eedcdf-3bf2-406c-a25e-1480bfa5452b tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "DELETE /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 204 len: 0 microversion: 2.1 time: 0.334760 May 28 03:48:52.453545 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 608/1174] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:48:52 2022] DELETE /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:48:52.459749 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-4c332ad1-7881-407d-8444-028d5f10cacb tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:48:52.460511 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-4c332ad1-7881-407d-8444-028d5f10cacb tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:48:52.467862 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c332ad1-7881-407d-8444-028d5f10cacb tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:52.468318 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c332ad1-7881-407d-8444-028d5f10cacb tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:52.469118 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c332ad1-7881-407d-8444-028d5f10cacb tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:52.469552 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-4c332ad1-7881-407d-8444-028d5f10cacb tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:52.604627 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-4c332ad1-7881-407d-8444-028d5f10cacb tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:48:52.612270 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-4c332ad1-7881-407d-8444-028d5f10cacb tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1578 microversion: 2.1 time: 0.157074 May 28 03:48:52.612848 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 567/1175] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:48:52 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1578 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 28 03:48:53.631384 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-1946e92a-2f76-496f-874f-404a8ca54774 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:48:53.632082 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-1946e92a-2f76-496f-874f-404a8ca54774 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:48:53.639223 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1946e92a-2f76-496f-874f-404a8ca54774 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:53.639670 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1946e92a-2f76-496f-874f-404a8ca54774 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:53.640438 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1946e92a-2f76-496f-874f-404a8ca54774 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:53.640902 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-1946e92a-2f76-496f-874f-404a8ca54774 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:53.771472 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-1946e92a-2f76-496f-874f-404a8ca54774 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:48:53.779655 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-1946e92a-2f76-496f-874f-404a8ca54774 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1578 microversion: 2.1 time: 0.152586 May 28 03:48:53.780306 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 609/1176] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:48:53 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1578 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:48:54.457705 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-d47bb16b-903a-4cec-8149-3cdf2761a50b req-a7408773-dbbd-41e5-8d0e-1d5e1d21ca9a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6ba87599-510f-4138-92f7-e909bb7df417", "name": "network-vif-deleted", "tag": "11df9db6-964f-4442-9b85-d9da3d305d3c"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:48:54.466390 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-d47bb16b-903a-4cec-8149-3cdf2761a50b req-a7408773-dbbd-41e5-8d0e-1d5e1d21ca9a service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:54.466836 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-d47bb16b-903a-4cec-8149-3cdf2761a50b req-a7408773-dbbd-41e5-8d0e-1d5e1d21ca9a service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:54.477912 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-d47bb16b-903a-4cec-8149-3cdf2761a50b req-a7408773-dbbd-41e5-8d0e-1d5e1d21ca9a service nova] Creating event network-vif-deleted:11df9db6-964f-4442-9b85-d9da3d305d3c for instance 6ba87599-510f-4138-92f7-e909bb7df417 on hv-cinder-84336 May 28 03:48:54.483424 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-d47bb16b-903a-4cec-8149-3cdf2761a50b req-a7408773-dbbd-41e5-8d0e-1d5e1d21ca9a service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029739 May 28 03:48:54.484003 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 568/1177] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:48:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:48:54.797563 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-6a95a96f-ccaf-43e5-9c1c-6a19de35a42e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:48:54.798185 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-6a95a96f-ccaf-43e5-9c1c-6a19de35a42e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:48:54.804789 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6a95a96f-ccaf-43e5-9c1c-6a19de35a42e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:54.805288 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6a95a96f-ccaf-43e5-9c1c-6a19de35a42e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:54.806062 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6a95a96f-ccaf-43e5-9c1c-6a19de35a42e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:54.806688 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-6a95a96f-ccaf-43e5-9c1c-6a19de35a42e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:54.874417 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-6a95a96f-ccaf-43e5-9c1c-6a19de35a42e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:48:54.882749 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-6a95a96f-ccaf-43e5-9c1c-6a19de35a42e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 200 len: 1226 microversion: 2.1 time: 0.089646 May 28 03:48:54.883316 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 610/1178] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:48:54 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 1226 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:48:55.901902 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-abf08122-9bb2-400f-b81d-64f3a30da090 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:48:55.903835 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-abf08122-9bb2-400f-b81d-64f3a30da090 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 6ba87599-510f-4138-92f7-e909bb7df417] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:48:55.910679 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-abf08122-9bb2-400f-b81d-64f3a30da090 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:55.911130 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-abf08122-9bb2-400f-b81d-64f3a30da090 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:55.911909 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-abf08122-9bb2-400f-b81d-64f3a30da090 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:55.912550 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-abf08122-9bb2-400f-b81d-64f3a30da090 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:55.926436 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-abf08122-9bb2-400f-b81d-64f3a30da090 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] HTTP exception thrown: Instance 6ba87599-510f-4138-92f7-e909bb7df417 could not be found. May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-abf08122-9bb2-400f-b81d-64f3a30da090 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Request method failure captured: May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 HTTP/1.1 May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZn0Qui27PFWt1C83-xg4yFPdjhPZnvWyBxCM-f8Qe2hd8rHuY3C9k6tR-bfxeJKyVsUF3_ynOM4ajKaj8L8A-8_VTS3F3IE_MTEvHpEo5Ag53NhTWKnyeMIUf8WBxjraePfkaBREJ6nRfxcVNjrDHcZ3Vo4xiTG2J-BAt0jGB4RLTo May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: c0fe831f6e954e05b7d448f1d4f43754 May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-TestVolumeBootPattern-1665013407 May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:48:55.928202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-TestVolumeBootPattern-1665013407 May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: c0fe831f6e954e05b7d448f1d4f43754 May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-TestVolumeBootPattern-1665013407 May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-TestVolumeBootPattern-1665013407-project May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 2b515cebf18643c38f1cfc2e3fad1db4 May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-TestVolumeBootPattern-1665013407-project May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 6ba87599-510f-4138-92f7-e909bb7df417 could not be found. May 28 03:48:55.929973 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '6ba87599-510f-4138-92f7-e909bb7df417'} May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6ba87599-510f-4138-92f7-e909bb7df417 could not be found. May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:48:55.930804 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6ba87599-510f-4138-92f7-e909bb7df417 could not be found. May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6ba87599-510f-4138-92f7-e909bb7df417 could not be found. May 28 03:48:55.932155 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:48:55.933471 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-abf08122-9bb2-400f-b81d-64f3a30da090 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Returning 404 to user: Instance 6ba87599-510f-4138-92f7-e909bb7df417 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:48:55.933471 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-abf08122-9bb2-400f-b81d-64f3a30da090 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417" status: 404 len: 111 microversion: 2.1 time: 0.032444 May 28 03:48:55.933471 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 569/1179] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:48:55 2022] GET /compute/v2.1/servers/6ba87599-510f-4138-92f7-e909bb7df417 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:48:59.253275 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-c1f09267-8dff-433c-9866-e1ca4c6d9532 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:48:59.257532 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c1f09267-8dff-433c-9866-e1ca4c6d9532 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:48:59.258110 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-f8089d1d-139f-4068-ba04-aad0ed7ac873 req-55681851-605e-4774-beec-85c9e662f008 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8c6232c7-4a11-4f67-a028-96d05393e4df", "tag": "1049955a-bd14-461f-b144-ea03d2f89fe6"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:48:59.261501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c1f09267-8dff-433c-9866-e1ca4c6d9532 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:59.261952 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c1f09267-8dff-433c-9866-e1ca4c6d9532 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:59.262736 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c1f09267-8dff-433c-9866-e1ca4c6d9532 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:59.263182 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-c1f09267-8dff-433c-9866-e1ca4c6d9532 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:59.276099 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-f8089d1d-139f-4068-ba04-aad0ed7ac873 req-55681851-605e-4774-beec-85c9e662f008 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:59.276677 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-f8089d1d-139f-4068-ba04-aad0ed7ac873 req-55681851-605e-4774-beec-85c9e662f008 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:59.290162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-f8089d1d-139f-4068-ba04-aad0ed7ac873 req-55681851-605e-4774-beec-85c9e662f008 service nova] Creating event network-changed:1049955a-bd14-461f-b144-ea03d2f89fe6 for instance 8c6232c7-4a11-4f67-a028-96d05393e4df on hv-cinder-84336 May 28 03:48:59.291282 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-c1f09267-8dff-433c-9866-e1ca4c6d9532 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Going to try to terminate instance {{(pid=124604) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:48:59.296469 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-f8089d1d-139f-4068-ba04-aad0ed7ac873 req-55681851-605e-4774-beec-85c9e662f008 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045307 May 28 03:48:59.297164 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 570/1180] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:48:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 28 03:48:59.624623 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-c1f09267-8dff-433c-9866-e1ca4c6d9532 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "DELETE /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 204 len: 0 microversion: 2.1 time: 0.375708 May 28 03:48:59.625824 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 611/1181] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:48:59 2022] DELETE /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 0 bytes in 378 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:48:59.634767 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-5e92f8be-5cfc-4bfa-a0d9-71175f80ae72 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:48:59.635963 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-5e92f8be-5cfc-4bfa-a0d9-71175f80ae72 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:48:59.643202 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e92f8be-5cfc-4bfa-a0d9-71175f80ae72 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:59.643918 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e92f8be-5cfc-4bfa-a0d9-71175f80ae72 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:48:59.644984 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e92f8be-5cfc-4bfa-a0d9-71175f80ae72 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:48:59.645479 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-5e92f8be-5cfc-4bfa-a0d9-71175f80ae72 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:00.079597 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-5e92f8be-5cfc-4bfa-a0d9-71175f80ae72 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:49:00.088029 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-5e92f8be-5cfc-4bfa-a0d9-71175f80ae72 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1577 microversion: 2.1 time: 0.457216 May 28 03:49:00.088778 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 571/1182] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:48:59 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1577 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:49:01.107039 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-b7e226cd-d395-4dde-b369-62f433630621 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:01.107662 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-b7e226cd-d395-4dde-b369-62f433630621 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:49:01.114045 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b7e226cd-d395-4dde-b369-62f433630621 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:01.116057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b7e226cd-d395-4dde-b369-62f433630621 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:01.116057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b7e226cd-d395-4dde-b369-62f433630621 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:01.116057 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-b7e226cd-d395-4dde-b369-62f433630621 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:01.246996 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-b7e226cd-d395-4dde-b369-62f433630621 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:49:01.255641 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-b7e226cd-d395-4dde-b369-62f433630621 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1458 microversion: 2.1 time: 0.153381 May 28 03:49:01.256493 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 612/1183] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:49:01 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1458 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:49:01.751531 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [req-ccd88ed0-5ab4-4336-a38f-3120544fb1de req-86bd4666-dd6f-4183-bc06-92a13c8ed466 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8c6232c7-4a11-4f67-a028-96d05393e4df", "name": "network-vif-deleted", "tag": "1049955a-bd14-461f-b144-ea03d2f89fe6"}]} {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:49:01.760052 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-ccd88ed0-5ab4-4336-a38f-3120544fb1de req-86bd4666-dd6f-4183-bc06-92a13c8ed466 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:01.766039 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [req-ccd88ed0-5ab4-4336-a38f-3120544fb1de req-86bd4666-dd6f-4183-bc06-92a13c8ed466 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:01.771160 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.compute.server_external_events [req-ccd88ed0-5ab4-4336-a38f-3120544fb1de req-86bd4666-dd6f-4183-bc06-92a13c8ed466 service nova] Creating event network-vif-deleted:1049955a-bd14-461f-b144-ea03d2f89fe6 for instance 8c6232c7-4a11-4f67-a028-96d05393e4df on hv-cinder-84336 May 28 03:49:01.776419 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [req-ccd88ed0-5ab4-4336-a38f-3120544fb1de req-86bd4666-dd6f-4183-bc06-92a13c8ed466 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029178 May 28 03:49:01.777573 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 572/1184] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:49:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:49:02.273239 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-3f59e8a3-9c6d-424c-a8e7-96c6de7ae26e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:02.273795 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-3f59e8a3-9c6d-424c-a8e7-96c6de7ae26e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:49:02.280808 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3f59e8a3-9c6d-424c-a8e7-96c6de7ae26e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:02.281239 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3f59e8a3-9c6d-424c-a8e7-96c6de7ae26e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:02.282001 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3f59e8a3-9c6d-424c-a8e7-96c6de7ae26e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:02.282441 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3f59e8a3-9c6d-424c-a8e7-96c6de7ae26e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:02.312790 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3f59e8a3-9c6d-424c-a8e7-96c6de7ae26e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:02.313284 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-3f59e8a3-9c6d-424c-a8e7-96c6de7ae26e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:02.313733 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-3f59e8a3-9c6d-424c-a8e7-96c6de7ae26e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lazy-loading 'fault' on Instance uuid 8c6232c7-4a11-4f67-a028-96d05393e4df {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:49:02.353671 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-3f59e8a3-9c6d-424c-a8e7-96c6de7ae26e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2b515cebf18643c38f1cfc2e3fad1db4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0fe831f6e954e05b7d448f1d4f43754', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:49:02.361592 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-3f59e8a3-9c6d-424c-a8e7-96c6de7ae26e tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 200 len: 1230 microversion: 2.1 time: 0.092327 May 28 03:49:02.362158 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 613/1185] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:49:02 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 1230 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:49:03.379465 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7cff32f3-7e38-473e-a34c-c5d3a75590aa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:03.380187 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-7cff32f3-7e38-473e-a34c-c5d3a75590aa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: 8c6232c7-4a11-4f67-a028-96d05393e4df] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:49:03.386942 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7cff32f3-7e38-473e-a34c-c5d3a75590aa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:03.387432 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7cff32f3-7e38-473e-a34c-c5d3a75590aa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:03.388762 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7cff32f3-7e38-473e-a34c-c5d3a75590aa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:03.389220 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-7cff32f3-7e38-473e-a34c-c5d3a75590aa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:03.401875 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-7cff32f3-7e38-473e-a34c-c5d3a75590aa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] HTTP exception thrown: Instance 8c6232c7-4a11-4f67-a028-96d05393e4df could not be found. May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7cff32f3-7e38-473e-a34c-c5d3a75590aa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Request method failure captured: May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df HTTP/1.1 May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZn0Qui27PFWt1C83-xg4yFPdjhPZnvWyBxCM-f8Qe2hd8rHuY3C9k6tR-bfxeJKyVsUF3_ynOM4ajKaj8L8A-8_VTS3F3IE_MTEvHpEo5Ag53NhTWKnyeMIUf8WBxjraePfkaBREJ6nRfxcVNjrDHcZ3Vo4xiTG2J-BAt0jGB4RLTo May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: c0fe831f6e954e05b7d448f1d4f43754 May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-TestVolumeBootPattern-1665013407 May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:49:03.404088 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-TestVolumeBootPattern-1665013407 May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: c0fe831f6e954e05b7d448f1d4f43754 May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-TestVolumeBootPattern-1665013407 May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-TestVolumeBootPattern-1665013407-project May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 2b515cebf18643c38f1cfc2e3fad1db4 May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-TestVolumeBootPattern-1665013407-project May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance 8c6232c7-4a11-4f67-a028-96d05393e4df could not be found. May 28 03:49:03.405761 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': '8c6232c7-4a11-4f67-a028-96d05393e4df'} May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8c6232c7-4a11-4f67-a028-96d05393e4df could not be found. May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:49:03.406874 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8c6232c7-4a11-4f67-a028-96d05393e4df could not be found. May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8c6232c7-4a11-4f67-a028-96d05393e4df could not be found. May 28 03:49:03.408420 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:03.410034 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-7cff32f3-7e38-473e-a34c-c5d3a75590aa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Returning 404 to user: Instance 8c6232c7-4a11-4f67-a028-96d05393e4df could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:49:03.410034 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-7cff32f3-7e38-473e-a34c-c5d3a75590aa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df" status: 404 len: 111 microversion: 2.1 time: 0.030472 May 28 03:49:03.410034 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 573/1186] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:49:03 2022] GET /compute/v2.1/servers/8c6232c7-4a11-4f67-a028-96d05393e4df => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:49:03.902097 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-37f274f8-8647-4f6c-8e48-62aca2a86584 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:03.902591 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-37f274f8-8647-4f6c-8e48-62aca2a86584 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:49:03.909399 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-37f274f8-8647-4f6c-8e48-62aca2a86584 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:03.909807 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-37f274f8-8647-4f6c-8e48-62aca2a86584 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:03.910759 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-37f274f8-8647-4f6c-8e48-62aca2a86584 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:03.911201 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-37f274f8-8647-4f6c-8e48-62aca2a86584 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:03.925134 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.wsgi [None req-37f274f8-8647-4f6c-8e48-62aca2a86584 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] HTTP exception thrown: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-37f274f8-8647-4f6c-8e48-62aca2a86584 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Request method failure captured: May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: request: DELETE /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 HTTP/1.1 May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept: application/json May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Accept-Encoding: identity May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Connection: close May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Length: 0 May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Content-Type: application/json May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Host: 10.222.0.41 May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Openstack-System-Scope: None May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: User-Agent: python-urllib3/1.26.9 May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Auth-Token: gAAAAABikZn0Qui27PFWt1C83-xg4yFPdjhPZnvWyBxCM-f8Qe2hd8rHuY3C9k6tR-bfxeJKyVsUF3_ynOM4ajKaj8L8A-8_VTS3F3IE_MTEvHpEo5Ag53NhTWKnyeMIUf8WBxjraePfkaBREJ6nRfxcVNjrDHcZ3Vo4xiTG2J-BAt0jGB4RLTo May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Id: None May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Domain-Name: None May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Identity-Status: Confirmed May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Is-Admin-Project: True May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Id: default May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Domain-Name: Default May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Id: c0fe831f6e954e05b7d448f1d4f43754 May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Project-Name: tempest-TestVolumeBootPattern-1665013407 May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Role: reader,member May 28 03:49:03.926857 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Roles: reader,member May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant: tempest-TestVolumeBootPattern-1665013407 May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Id: c0fe831f6e954e05b7d448f1d4f43754 May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-Tenant-Name: tempest-TestVolumeBootPattern-1665013407 May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User: tempest-TestVolumeBootPattern-1665013407-project May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Id: default May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Domain-Name: Default May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Id: 2b515cebf18643c38f1cfc2e3fad1db4 May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: X-User-Name: tempest-TestVolumeBootPattern-1665013407-project May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: method: > May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: exception: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:49:03.928474 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: action_args: {'id': 'b53491f6-a319-4950-ac9e-a9b0bd7a45e2'} May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:49:03.929501 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:49:03.930816 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR nova.api.openstack.wsgi May 28 03:49:03.932247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-37f274f8-8647-4f6c-8e48-62aca2a86584 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Returning 404 to user: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. {{(pid=124604) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:49:03.932247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-37f274f8-8647-4f6c-8e48-62aca2a86584 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "DELETE /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 404 len: 111 microversion: 2.1 time: 0.029992 May 28 03:49:03.932247 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 614/1187] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:49:03 2022] DELETE /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:49:03.937393 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0269516b-376b-4159-86a7-f7d93c8f2660 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:03.938037 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-0269516b-376b-4159-86a7-f7d93c8f2660 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [instance: b53491f6-a319-4950-ac9e-a9b0bd7a45e2] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:49:03.943940 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0269516b-376b-4159-86a7-f7d93c8f2660 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:03.944385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0269516b-376b-4159-86a7-f7d93c8f2660 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:03.945162 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0269516b-376b-4159-86a7-f7d93c8f2660 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:03.945596 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-0269516b-376b-4159-86a7-f7d93c8f2660 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:03.958529 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-0269516b-376b-4159-86a7-f7d93c8f2660 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] HTTP exception thrown: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0269516b-376b-4159-86a7-f7d93c8f2660 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Request method failure captured: May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 HTTP/1.1 May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZn0Qui27PFWt1C83-xg4yFPdjhPZnvWyBxCM-f8Qe2hd8rHuY3C9k6tR-bfxeJKyVsUF3_ynOM4ajKaj8L8A-8_VTS3F3IE_MTEvHpEo5Ag53NhTWKnyeMIUf8WBxjraePfkaBREJ6nRfxcVNjrDHcZ3Vo4xiTG2J-BAt0jGB4RLTo May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: c0fe831f6e954e05b7d448f1d4f43754 May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-TestVolumeBootPattern-1665013407 May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:49:03.960004 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_c0fe831f6e954e05b7d448f1d4f43754"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-TestVolumeBootPattern-1665013407 May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: c0fe831f6e954e05b7d448f1d4f43754 May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-TestVolumeBootPattern-1665013407 May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-TestVolumeBootPattern-1665013407-project May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 2b515cebf18643c38f1cfc2e3fad1db4 May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-TestVolumeBootPattern-1665013407-project May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:49:03.961692 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': 'b53491f6-a319-4950-ac9e-a9b0bd7a45e2'} May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:49:03.962720 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. May 28 03:49:03.964254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:03.965898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-0269516b-376b-4159-86a7-f7d93c8f2660 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Returning 404 to user: Instance b53491f6-a319-4950-ac9e-a9b0bd7a45e2 could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:49:03.965898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-0269516b-376b-4159-86a7-f7d93c8f2660 tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2" status: 404 len: 111 microversion: 2.1 time: 0.028402 May 28 03:49:03.965898 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 574/1188] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:49:03 2022] GET /compute/v2.1/servers/b53491f6-a319-4950-ac9e-a9b0bd7a45e2 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:49:05.433999 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-db749d65-80de-4e52-ac97-27649eebc2fa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] Calling method '.version_select at 0x7feb1f3be4c0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:05.439128 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: ERROR oslo.messaging._drivers.impl_rabbit [None req-db749d65-80de-4e52-ac97-27649eebc2fa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [3c24e054-72b1-4d40-a4d8-5c58440a3d57] AMQP server on 10.222.0.41:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 28 03:49:06.445212 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO oslo.messaging._drivers.impl_rabbit [None req-db749d65-80de-4e52-ac97-27649eebc2fa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] [3c24e054-72b1-4d40-a4d8-5c58440a3d57] Reconnected to AMQP server on 10.222.0.41:5672 via [amqp] client with port 37286. May 28 03:49:06.527440 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-db749d65-80de-4e52-ac97-27649eebc2fa tempest-TestVolumeBootPattern-1665013407 tempest-TestVolumeBootPattern-1665013407-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1597361276" status: 202 len: 0 microversion: 2.1 time: 1.097447 May 28 03:49:06.528010 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 615/1189] 10.222.0.41 () {58 vars in 1262 bytes} [Sat May 28 03:49:05 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1597361276 => generated 0 bytes in 1099 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 28 03:49:11.591776 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-bd1250ee-b255-4245-8029-c404affc349a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:11.593127 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-bd1250ee-b255-4245-8029-c404affc349a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:49:11.609859 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bd1250ee-b255-4245-8029-c404affc349a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:11.611177 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bd1250ee-b255-4245-8029-c404affc349a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:11.613936 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bd1250ee-b255-4245-8029-c404affc349a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:11.616864 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-bd1250ee-b255-4245-8029-c404affc349a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:11.666826 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-bd1250ee-b255-4245-8029-c404affc349a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Going to try to terminate instance {{(pid=124605) delete /opt/stack/nova/nova/compute/api.py:2664}} May 28 03:49:11.758809 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-8840d5ca-c2ef-4aa0-8f3e-e55498bae879 req-12a69eca-c113-433a-848e-7143fd156374 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b37b6f57-630a-4342-bd89-6ccba324b98f", "tag": "791555af-21e2-4f77-8de6-77284ba93aae"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:49:11.768587 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-8840d5ca-c2ef-4aa0-8f3e-e55498bae879 req-12a69eca-c113-433a-848e-7143fd156374 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:11.769303 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-8840d5ca-c2ef-4aa0-8f3e-e55498bae879 req-12a69eca-c113-433a-848e-7143fd156374 service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:11.781652 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-8840d5ca-c2ef-4aa0-8f3e-e55498bae879 req-12a69eca-c113-433a-848e-7143fd156374 service nova] Creating event network-changed:791555af-21e2-4f77-8de6-77284ba93aae for instance b37b6f57-630a-4342-bd89-6ccba324b98f on hv-cinder-84336 May 28 03:49:11.787159 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-8840d5ca-c2ef-4aa0-8f3e-e55498bae879 req-12a69eca-c113-433a-848e-7143fd156374 service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.202306 May 28 03:49:11.788461 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 616/1190] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:49:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:49:12.115665 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-bd1250ee-b255-4245-8029-c404affc349a tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "DELETE /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 204 len: 0 microversion: 2.1 time: 0.533418 May 28 03:49:12.119054 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 575/1191] 10.222.0.41 () {58 vars in 1238 bytes} [Sat May 28 03:49:11 2022] DELETE /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 0 bytes in 535 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 28 03:49:12.129031 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-bcc7610b-163c-4b13-ae76-acdbb0592e96 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:12.130118 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-bcc7610b-163c-4b13-ae76-acdbb0592e96 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:49:12.138451 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bcc7610b-163c-4b13-ae76-acdbb0592e96 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:12.139213 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bcc7610b-163c-4b13-ae76-acdbb0592e96 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:12.140297 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bcc7610b-163c-4b13-ae76-acdbb0592e96 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:12.140990 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-bcc7610b-163c-4b13-ae76-acdbb0592e96 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:12.286238 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-bcc7610b-163c-4b13-ae76-acdbb0592e96 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:49:12.294789 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-bcc7610b-163c-4b13-ae76-acdbb0592e96 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.173592 May 28 03:49:12.300850 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 617/1192] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:49:12 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:49:13.313227 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-d3f592bb-0d93-4b8b-84ea-0d6582bc467e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:13.314537 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-d3f592bb-0d93-4b8b-84ea-0d6582bc467e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:49:13.322760 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3f592bb-0d93-4b8b-84ea-0d6582bc467e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:13.323575 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3f592bb-0d93-4b8b-84ea-0d6582bc467e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:13.324506 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3f592bb-0d93-4b8b-84ea-0d6582bc467e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:13.325174 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-d3f592bb-0d93-4b8b-84ea-0d6582bc467e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:13.484955 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-d3f592bb-0d93-4b8b-84ea-0d6582bc467e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:49:13.493563 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-d3f592bb-0d93-4b8b-84ea-0d6582bc467e tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1570 microversion: 2.1 time: 0.185174 May 28 03:49:13.494160 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 576/1193] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:49:13 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1570 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:49:14.417374 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [req-f8f040e0-6675-424c-9e8d-998823852d10 req-cc4887e2-d8b9-408b-9dad-eb7301263f4a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b37b6f57-630a-4342-bd89-6ccba324b98f", "name": "network-vif-deleted", "tag": "791555af-21e2-4f77-8de6-77284ba93aae"}]} {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 28 03:49:14.426919 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-f8f040e0-6675-424c-9e8d-998823852d10 req-cc4887e2-d8b9-408b-9dad-eb7301263f4a service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:14.427402 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [req-f8f040e0-6675-424c-9e8d-998823852d10 req-cc4887e2-d8b9-408b-9dad-eb7301263f4a service nova] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:14.440462 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.compute.server_external_events [req-f8f040e0-6675-424c-9e8d-998823852d10 req-cc4887e2-d8b9-408b-9dad-eb7301263f4a service nova] Creating event network-vif-deleted:791555af-21e2-4f77-8de6-77284ba93aae for instance b37b6f57-630a-4342-bd89-6ccba324b98f on hv-cinder-84336 May 28 03:49:14.445584 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [req-f8f040e0-6675-424c-9e8d-998823852d10 req-cc4887e2-d8b9-408b-9dad-eb7301263f4a service nova] 10.222.0.41 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031963 May 28 03:49:14.446182 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 618/1194] 10.222.0.41 () {64 vars in 1317 bytes} [Sat May 28 03:49:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 28 03:49:14.511154 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-68242c6f-511c-4f6c-9696-554607f4456c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:14.513451 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-68242c6f-511c-4f6c-9696-554607f4456c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:49:14.518469 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-68242c6f-511c-4f6c-9696-554607f4456c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:14.518919 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-68242c6f-511c-4f6c-9696-554607f4456c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:14.519698 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-68242c6f-511c-4f6c-9696-554607f4456c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:14.520137 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-68242c6f-511c-4f6c-9696-554607f4456c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:14.591855 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.policy [None req-68242c6f-511c-4f6c-9696-554607f4456c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124605) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:49:14.600781 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-68242c6f-511c-4f6c-9696-554607f4456c tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1383 microversion: 2.1 time: 0.093278 May 28 03:49:14.601659 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 577/1195] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:49:14 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1383 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:49:15.619090 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-f04d8082-d4b7-43f8-b146-85379d5fd0a8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:15.619792 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.compute.api [None req-f04d8082-d4b7-43f8-b146-85379d5fd0a8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124604) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:49:15.626439 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f04d8082-d4b7-43f8-b146-85379d5fd0a8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:15.626887 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f04d8082-d4b7-43f8-b146-85379d5fd0a8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:15.627860 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f04d8082-d4b7-43f8-b146-85379d5fd0a8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:15.628294 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f04d8082-d4b7-43f8-b146-85379d5fd0a8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:15.660026 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f04d8082-d4b7-43f8-b146-85379d5fd0a8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:15.660483 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG oslo_concurrency.lockutils [None req-f04d8082-d4b7-43f8-b146-85379d5fd0a8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124604) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:15.660930 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.objects.instance [None req-f04d8082-d4b7-43f8-b146-85379d5fd0a8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lazy-loading 'fault' on Instance uuid b37b6f57-630a-4342-bd89-6ccba324b98f {{(pid=124604) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 28 03:49:15.705246 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.policy [None req-f04d8082-d4b7-43f8-b146-85379d5fd0a8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303285e036734e5cbd491920ae7473d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd1d9f51a3cc44062af827a498842ee80', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124604) authorize /opt/stack/nova/nova/policy.py:199}} May 28 03:49:15.713242 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-f04d8082-d4b7-43f8-b146-85379d5fd0a8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 200 len: 1225 microversion: 2.1 time: 0.098703 May 28 03:49:15.713822 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 619/1196] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:49:15 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 28 03:49:16.730755 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-984916c9-efa9-4ee7-beaa-d608bb0f17d7 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '>' {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:16.731846 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.compute.api [None req-984916c9-efa9-4ee7-beaa-d608bb0f17d7 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] [instance: b37b6f57-630a-4342-bd89-6ccba324b98f] Fetching instance by UUID {{(pid=124605) get /opt/stack/nova/nova/compute/api.py:2920}} May 28 03:49:16.738117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-984916c9-efa9-4ee7-beaa-d608bb0f17d7 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:16.738559 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-984916c9-efa9-4ee7-beaa-d608bb0f17d7 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:16.739309 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-984916c9-efa9-4ee7-beaa-d608bb0f17d7 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 28 03:49:16.739739 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG oslo_concurrency.lockutils [None req-984916c9-efa9-4ee7-beaa-d608bb0f17d7 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Lock "023203ac-1a27-4ff9-b011-8825c729816d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 28 03:49:16.753646 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.wsgi [None req-984916c9-efa9-4ee7-beaa-d608bb0f17d7 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] HTTP exception thrown: Instance b37b6f57-630a-4342-bd89-6ccba324b98f could not be found. May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-984916c9-efa9-4ee7-beaa-d608bb0f17d7 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Request method failure captured: May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: request: GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f HTTP/1.1 May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept: application/json May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Accept-Encoding: identity May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Connection: close May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Length: 0 May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Content-Type: application/json May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Host: 10.222.0.41 May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Openstack-System-Scope: None May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: User-Agent: python-urllib3/1.26.9 May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Auth-Token: gAAAAABikZqo_i-U6X4Tf4EYQFqJiwCumzolV4QFepna0j5cajn7RyVyl8bSryBoOoGarEMBD_sjAxdIwQBFj_aehxcvKF00T64hYhijPv036iXQuXlXTBIf3wgl1Ms1Grdv1XQjt-bPgwMpQPjfh_VCVOI-P07XX7V4z5yoBLh1-Sz_-cNlQnk May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Id: None May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Domain-Name: None May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Identity-Status: Confirmed May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Is-Admin-Project: True May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Id: default May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Domain-Name: Default May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Id: d1d9f51a3cc44062af827a498842ee80 May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Project-Name: tempest-TestShelveInstance-2046420353 May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Role: reader,member May 28 03:49:16.755254 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Roles: reader,member May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2/d1d9f51a3cc44062af827a498842ee80"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/d1d9f51a3cc44062af827a498842ee80"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41:9696/"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/volume/v3/d1d9f51a3cc44062af827a498842ee80"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.41:8080", "publicURL": "http://10.222.0.41:8080/v1/AUTH_d1d9f51a3cc44062af827a498842ee80"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/placement"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.41/image"}]}] May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant: tempest-TestShelveInstance-2046420353 May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Id: d1d9f51a3cc44062af827a498842ee80 May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-Tenant-Name: tempest-TestShelveInstance-2046420353 May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User: tempest-TestShelveInstance-2046420353-project May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Id: default May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Domain-Name: Default May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Id: 303285e036734e5cbd491920ae7473d9 May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: X-User-Name: tempest-TestShelveInstance-2046420353-project May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: method: > May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: exception: Instance b37b6f57-630a-4342-bd89-6ccba324b98f could not be found. May 28 03:49:16.757216 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: action_args: {'id': 'b37b6f57-630a-4342-bd89-6ccba324b98f'} May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: {{(pid=124605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b37b6f57-630a-4342-bd89-6ccba324b98f could not be found. May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 28 03:49:16.758385 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b37b6f57-630a-4342-bd89-6ccba324b98f could not be found. May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b37b6f57-630a-4342-bd89-6ccba324b98f could not be found. May 28 03:49:16.760117 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: ERROR nova.api.openstack.wsgi May 28 03:49:16.761652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: DEBUG nova.api.openstack.wsgi [None req-984916c9-efa9-4ee7-beaa-d608bb0f17d7 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Returning 404 to user: Instance b37b6f57-630a-4342-bd89-6ccba324b98f could not be found. {{(pid=124605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 28 03:49:16.761652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO nova.api.openstack.requestlog [None req-984916c9-efa9-4ee7-beaa-d608bb0f17d7 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f" status: 404 len: 111 microversion: 2.1 time: 0.029795 May 28 03:49:16.761652 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: [pid: 124605|app: 0|req: 578/1197] 10.222.0.41 () {58 vars in 1235 bytes} [Sat May 28 03:49:16 2022] GET /compute/v2.1/servers/b37b6f57-630a-4342-bd89-6ccba324b98f => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 28 03:49:16.966430 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: DEBUG nova.api.openstack.wsgi [None req-a15e8cde-44ff-459d-8c14-ffdf1c8694e8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] Calling method '.version_select at 0x7feb310221f0>' {{(pid=124604) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 28 03:49:17.070399 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO nova.api.openstack.requestlog [None req-a15e8cde-44ff-459d-8c14-ffdf1c8694e8 tempest-TestShelveInstance-2046420353 tempest-TestShelveInstance-2046420353-project] 10.222.0.41 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1367140617" status: 202 len: 0 microversion: 2.1 time: 0.108396 May 28 03:49:17.071034 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: [pid: 124604|app: 0|req: 620/1198] 10.222.0.41 () {58 vars in 1253 bytes} [Sat May 28 03:49:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1367140617 => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 28 03:49:33.641773 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:49:33.642547 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Sat May 28 03:49:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:50:51.497590 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:50:51.498182 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Sat May 28 03:50:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:51:02.787945 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:51:02.788657 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Sat May 28 03:51:02 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:51:15.804499 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:51:15.805086 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Sat May 28 03:51:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:51:19.740929 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:51:19.741546 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Sat May 28 03:51:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:52:32.455379 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:52:32.455981 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Sat May 28 03:52:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:52:53.649709 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:52:53.650460 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Sat May 28 03:52:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:54:11.505721 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:54:11.506335 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Sat May 28 03:54:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:54:22.795837 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:54:22.796455 dv-cinder-843361-3-iscsi devstack@n-api.service[124605]: Sat May 28 03:54:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 28 03:54:35.812491 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 28 03:54:35.813045 dv-cinder-843361-3-iscsi devstack@n-api.service[124604]: Sat May 28 03:54:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!